{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "826ca0dcff42a3ba",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Multi-Information Source BO with Augmented Gaussian Processes\n",
    "- Contributors: andreaponti5\n",
    "- Last updated: Jan 29, 2024\n",
    "- BoTorch version: 0.9.5(dev)\n",
    "\n",
    "In this tutorial, we show how to perform Multiple Information Source Bayesian Optimization in BoTorch based on the Augmented Gaussian Process (AGP) and the Augmented UCB (AUCB) acquisition function proposed in [1].\n",
    "The key idea of the AGP is to fit a GP model for each information source and *augment* the observations on the high fidelity source with those from *cheaper* sources which can be considered as *reliable*. The GP model fitted on this *augmented* set of observations is the AGP.\n",
    "The AUCB is a modification of the standard UCB -- computed on the AGP -- suitably proposed to also deal with the source-specific query cost.\n",
    "\n",
    "We emprically show that the *AGP-based* Multiple Information Source Basyesian Optimization usually performs better than other multi-fidelity approaches [2].\n",
    "\n",
    "[1] [Candelieri, A., & Archetti, F. (2021). Sparsifying to optimize over multiple information sources: an augmented Gaussian process based algorithm. Structural and Multidisciplinary Optimization, 64, 239-255.](https://link.springer.com/article/10.1007/s00158-021-02882-7)\n",
    "[2] [The arxiv will be available soon.](https://arxiv.org/)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8aa9032dbb2c2b04",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:54:55.714265100Z",
     "start_time": "2024-02-08T07:54:52.594530700Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!pip install matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e55defd1ee4a5b0f",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:55:06.365087700Z",
     "start_time": "2024-02-08T07:54:55.714265100Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import torch\n",
    "from gpytorch import ExactMarginalLogLikelihood\n",
    "\n",
    "import botorch\n",
    "from botorch import fit_gpytorch_mll\n",
    "from botorch.acquisition import InverseCostWeightedUtility, qMultiFidelityMaxValueEntropy\n",
    "from botorch_community.acquisition.augmented_multisource import AugmentedUpperConfidenceBound\n",
    "from botorch.models import AffineFidelityCostModel, SingleTaskMultiFidelityGP\n",
    "from botorch_community.models.gp_regression_multisource import SingleTaskAugmentedGP, get_random_x_for_agp\n",
    "from botorch.models.transforms import Standardize\n",
    "from botorch.optim import optimize_acqf, optimize_acqf_mixed\n",
    "from botorch.test_functions.multi_fidelity import AugmentedBranin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "initial_id",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:55:07.689089500Z",
     "start_time": "2024-02-08T07:55:06.361808100Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tkwargs = {\n",
    "    \"dtype\": torch.double,\n",
    "    \"device\": torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\"),\n",
    "}\n",
    "SMOKE_TEST = os.environ.get(\"SMOKE_TEST\", False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e316bd291459a135",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:55:07.701088200Z",
     "start_time": "2024-02-08T07:55:07.690705Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "N_ITER = 10 if SMOKE_TEST else 50\n",
    "SEED = 3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b58c67f5dbf329c",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Problem setup\n",
    "We consider the augmented Branin multi-fidelity synthetic test problem. It is important to clarify that *augmented* is not about the AGP: here, it has a different meaning. It means that the Branin test function has been modified by introducing an additional dimension representing the fidelity parameter.\n",
    "\n",
    "The test function takes the form $f(x,s)$ where $x \\in [-5, 10] \\times [0, 15]$ and $s \\in [0,1]$. The target fidelity is 1.0, which means that our goal is to solve $\\max_x f(x,1.0)$ by making use of cheaper evaluations $f(x,s)$ for $s < 1.0$. In this example, we'll assume that the cost function takes the form $5.0 + s$, illustrating a situation where the fixed cost is $5.0$.\n",
    "\n",
    "Since a multiple information source context is considered, three different sources are considered, with $s = 0.5, 0.75, 1.00$, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "5f13380e681011ea",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:55:07.793085900Z",
     "start_time": "2024-02-08T07:55:07.698958300Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "problem = AugmentedBranin(negate=True).to(**tkwargs)\n",
    "fidelities = torch.tensor([0.5, 0.75, 1.0], **tkwargs)\n",
    "n_sources = fidelities.shape[0]\n",
    "\n",
    "bounds = torch.tensor([[-5, 0, 0], [10, 15, n_sources - 1]], **tkwargs)\n",
    "target_fidelities = {n_sources - 1: 1.0}\n",
    "\n",
    "cost_model = AffineFidelityCostModel(fidelity_weights=target_fidelities, fixed_cost=5.0)\n",
    "cost_aware_utility = InverseCostWeightedUtility(cost_model=cost_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81e30344694a9583",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Model initialization\n",
    "\n",
    "We use a `SingleTaskAugmentedGP` to implement our AGP.\n",
    "\n",
    "At each Bayesian Optimization iteration, the set of observations from the *ground-truth* (i.e., the highest fidelity and more expensive source) is temporarily *augmented* by including observations from the other cheap sources, only if they can be considered *reliable*. Specifically, an observation $(x,y)$ from a cheap source is considered reliable if it satisfies the following inequality:\n",
    "\n",
    "$$\\vert\\mu(x)-y\\vert \\leq m \\sigma(x)$$\n",
    "\n",
    "where $\\mu(x)$ and $\\sigma(x)$ are, respectively, the posterior mean and standard deviation of the GP model fitted on the high fidelity observations only, and $m$ is a technical parameter making more *conservative* ($m→0$) or *inclusive* ($m→∞)$ the augmentation process. As reported in [1], a suitable value for this parameter is $m=1$.\n",
    "\n",
    "After the set of observations is augmented, the AGP is fitted through `SingleTaskAugmentedGP`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f8272160f69227ef",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:55:07.801808200Z",
     "start_time": "2024-02-08T07:55:07.797392200Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def generate_initial_data(n):\n",
    "    train_x = get_random_x_for_agp(n, bounds, 1)\n",
    "    xs = train_x[..., :-1]\n",
    "    fids = fidelities[train_x[..., -1].int()].reshape(-1, 1)\n",
    "    train_obj = problem(torch.cat((xs, fids), dim=1)).unsqueeze(-1)\n",
    "    return train_x, train_obj\n",
    "\n",
    "\n",
    "def initialize_model(train_x, train_obj, m):\n",
    "    model = SingleTaskAugmentedGP(\n",
    "        train_x, train_obj, m=m, outcome_transform=Standardize(m=1),\n",
    "    )\n",
    "    mll = ExactMarginalLogLikelihood(model.likelihood, model)\n",
    "    return mll, model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad21cd7999805d78",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### Define a helper function that performs the essential BO step\n",
    "This helper function optimizes the acquisition function and returns the candidate point along with the observed function values.\n",
    "\n",
    "The UCB acquisition function has been modified to deal with both the *discrepancy* between information sources and the *source-specific query cost*.\n",
    "\n",
    "Formally, the AUCB acquisition function, at a generic iteration $t$, is defined as:\n",
    "\n",
    "$$\\alpha_s(x,\\hat y^+) = \\frac{\\left[\\hat{\\mu}(x) + \\sqrt{\\beta^{(t)}} \\hat{\\sigma}(x)\\right] - \\hat{y}^+}{c_s \\cdot (1+\\vert \\hat{\\mu}(x) - \\mu_s(x) \\vert)} $$\n",
    "\n",
    "where $\\hat{y}^+$ is the best (i.e., highest) value in the *augmented* set of observations, the numerator is -- therefore -- the optimistic improvement with respect to $\\hat{y}^+$, $c_s$ is the query cost for the source $s$, and $\\vert \\hat{\\mu}(x) - \\mu_s(x) \\vert$ is a discrepancy measure between the predictions provided by the AGP and the GP on the source $s$, respectively, given the input $x$ (i.e., 1 is added just to avoid division by zero).\n",
    "\n",
    "For more information, please refer to [1],"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "311309bd6a4d3a92",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:55:07.811037100Z",
     "start_time": "2024-02-08T07:55:07.802897300Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def optimize_aucb(acqf):\n",
    "    candidate, value = optimize_acqf(\n",
    "        acq_function=acqf,\n",
    "        bounds=bounds,\n",
    "        q=1,\n",
    "        num_restarts=5,\n",
    "        raw_samples=128,\n",
    "    )\n",
    "    # observe new values\n",
    "    new_x = candidate.detach()\n",
    "    new_x[:, -1] = torch.round(new_x[:, -1], decimals=0)\n",
    "    return new_x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "667b55ca7ae58af3",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Perform a few steps of multi-fidelity BO\n",
    "First, let's generate some initial random data and fit a surrogate model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a2b9dbfbae2f7d5",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:55:10.200578200Z",
     "start_time": "2024-02-08T07:55:07.808815500Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "torch.manual_seed(SEED)\n",
    "train_x, train_obj = generate_initial_data(n=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca54230c1481ba60",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "We can now use the helper functions above to run a few iterations of BO."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8d02e319798a28d7",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:59:16.933226800Z",
     "start_time": "2024-02-08T07:55:10.205902900Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 0;\t Fid = 1.00;\t Obj = -200.3252;\n",
      "Iter 1;\t Fid = 1.00;\t Obj = -38.1094;\n",
      "Iter 2;\t Fid = 1.00;\t Obj = -38.1090;\n",
      "Iter 3;\t Fid = 1.00;\t Obj = -38.1093;\n",
      "Iter 4;\t Fid = 1.00;\t Obj = -38.1093;\n",
      "Iter 5;\t Fid = 0.75;\t Obj = -10.3835;\n",
      "Iter 6;\t Fid = 1.00;\t Obj = -38.1093;\n",
      "Iter 7;\t Fid = 1.00;\t Obj = -36.9691;\n",
      "Iter 8;\t Fid = 1.00;\t Obj = -38.0601;\n",
      "Iter 9;\t Fid = 1.00;\t Obj = -36.4893;\n",
      "Iter 10;\t Fid = 1.00;\t Obj = -34.5676;\n",
      "Iter 11;\t Fid = 1.00;\t Obj = -27.2647;\n",
      "Iter 12;\t Fid = 1.00;\t Obj = -23.7780;\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\ponti\\Desktop\\workspace\\botorch\\botorch\\optim\\optimize.py:367: RuntimeWarning: Optimization failed in `gen_candidates_scipy` with the following warning(s):\n",
      "[OptimizationWarning('Optimization failed within `scipy.optimize.minimize` with status 2 and message ABNORMAL_TERMINATION_IN_LNSRCH.')]\n",
      "Trying again with a new set of initial conditions.\n",
      "  warnings.warn(first_warn_msg, RuntimeWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 13;\t Fid = 1.00;\t Obj = -50.3585;\n",
      "Iter 14;\t Fid = 1.00;\t Obj = -35.5112;\n",
      "Iter 15;\t Fid = 1.00;\t Obj = -17.9104;\n",
      "Iter 16;\t Fid = 1.00;\t Obj = -12.9752;\n",
      "Iter 17;\t Fid = 1.00;\t Obj = -13.3390;\n",
      "Iter 18;\t Fid = 1.00;\t Obj = -8.9100;\n",
      "Iter 19;\t Fid = 1.00;\t Obj = -3.2656;\n",
      "Iter 20;\t Fid = 1.00;\t Obj = -2.7450;\n",
      "Iter 21;\t Fid = 1.00;\t Obj = -5.7930;\n",
      "Iter 22;\t Fid = 1.00;\t Obj = -0.6055;\n",
      "Iter 23;\t Fid = 1.00;\t Obj = -0.5995;\n",
      "Iter 24;\t Fid = 1.00;\t Obj = -2.8311;\n",
      "Iter 25;\t Fid = 1.00;\t Obj = -1.0225;\n",
      "Iter 26;\t Fid = 1.00;\t Obj = -2.8477;\n",
      "Iter 27;\t Fid = 1.00;\t Obj = -1.0333;\n",
      "Iter 28;\t Fid = 1.00;\t Obj = -0.4881;\n",
      "Iter 29;\t Fid = 1.00;\t Obj = -3.4325;\n",
      "Iter 30;\t Fid = 1.00;\t Obj = -0.8336;\n",
      "Iter 31;\t Fid = 1.00;\t Obj = -0.4630;\n",
      "Iter 32;\t Fid = 1.00;\t Obj = -2.3562;\n",
      "Iter 33;\t Fid = 1.00;\t Obj = -4.9150;\n",
      "Iter 34;\t Fid = 1.00;\t Obj = -0.4038;\n",
      "Iter 35;\t Fid = 1.00;\t Obj = -1.9965;\n",
      "Iter 36;\t Fid = 1.00;\t Obj = -0.5458;\n",
      "Iter 37;\t Fid = 1.00;\t Obj = -5.4650;\n",
      "Iter 38;\t Fid = 1.00;\t Obj = -0.8234;\n",
      "Iter 39;\t Fid = 1.00;\t Obj = -0.6905;\n",
      "Iter 40;\t Fid = 1.00;\t Obj = -0.8766;\n",
      "Iter 41;\t Fid = 1.00;\t Obj = -0.9116;\n",
      "Iter 42;\t Fid = 1.00;\t Obj = -3.7283;\n",
      "Iter 43;\t Fid = 1.00;\t Obj = -1.8566;\n",
      "Iter 44;\t Fid = 1.00;\t Obj = -1.5902;\n",
      "Iter 45;\t Fid = 1.00;\t Obj = -1.2975;\n",
      "Iter 46;\t Fid = 1.00;\t Obj = -1.7442;\n",
      "Iter 47;\t Fid = 1.00;\t Obj = -6.0570;\n",
      "Iter 48;\t Fid = 1.00;\t Obj = -2.5479;\n",
      "Iter 49;\t Fid = 1.00;\t Obj = -1.4998;\n"
     ]
    }
   ],
   "source": [
    "cumulative_cost = 0.0\n",
    "\n",
    "with botorch.settings.validate_input_scaling(False):\n",
    "    for it in range(N_ITER):\n",
    "        mll, model = initialize_model(train_x, train_obj, m=1)\n",
    "        fit_gpytorch_mll(mll)\n",
    "        acqf = AugmentedUpperConfidenceBound(\n",
    "            model,\n",
    "            beta=3,\n",
    "            maximize=True,\n",
    "            best_f=train_obj[torch.where(train_x[:, -1] == 0)].min(),\n",
    "            cost={i: fid + 5.0 for i, fid in enumerate(fidelities)},\n",
    "        )\n",
    "        new_x = optimize_aucb(acqf)\n",
    "        if model.n_true_points < model.max_n_cheap_points:\n",
    "            new_x[:, -1] = fidelities.shape[0] - 1\n",
    "        train_x = torch.cat([train_x, new_x])\n",
    "\n",
    "        new_x[:, -1] = fidelities[new_x[:, -1].int()]\n",
    "        new_obj = problem(new_x).unsqueeze(-1)\n",
    "        train_obj = torch.cat([train_obj, new_obj])\n",
    "\n",
    "        print(\n",
    "            f\"Iter {it};\"\n",
    "            f\"\\t Fid = {new_x[0].tolist()[-1]:.2f};\"\n",
    "            f\"\\t Obj = {new_obj[0][0].tolist():.4f};\"\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab5e775efb0ccfe9",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Comparison to MES"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1c93f20bdaffccac",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:59:16.933226800Z",
     "start_time": "2024-02-08T07:59:16.930006100Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def initialize_mes_model(train_x, train_obj, data_fidelity):\n",
    "    model = SingleTaskMultiFidelityGP(\n",
    "        train_x,\n",
    "        train_obj,\n",
    "        outcome_transform=Standardize(m=1),\n",
    "        data_fidelity=data_fidelity,\n",
    "    )\n",
    "    mll = ExactMarginalLogLikelihood(model.likelihood, model)\n",
    "    return mll, model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "1a1bba8e3496b54d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:59:16.943971Z",
     "start_time": "2024-02-08T07:59:16.935329500Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def optimize_mes_and_get_observation(mes_acq, fixed_features_list):\n",
    "    candidates, acq_value = optimize_acqf_mixed(\n",
    "        acq_function=mes_acq,\n",
    "        bounds=problem.bounds,\n",
    "        q=1,\n",
    "        num_restarts=5,\n",
    "        raw_samples=128,\n",
    "        fixed_features_list=fixed_features_list,\n",
    "    )\n",
    "    # observe new values\n",
    "    cost = cost_model(candidates).sum()\n",
    "    new_x = candidates.detach()\n",
    "    new_obj = problem(new_x).unsqueeze(-1)\n",
    "    return new_x, new_obj, cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e8414dfbd0643afa",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T07:59:16.963122100Z",
     "start_time": "2024-02-08T07:59:16.943971Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "train_x_mes = torch.clone(train_x[:10])\n",
    "train_x_mes[:, -1] = fidelities[train_x_mes[:, -1].int()]\n",
    "train_obj_mes = torch.clone(train_obj[:10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "fcaf20bf41f1b680",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T08:03:49.181588500Z",
     "start_time": "2024-02-08T07:59:16.951110100Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 0;\t Fid = 0.50;\t Obj = -85.5566;\n",
      "Iter 1;\t Fid = 0.50;\t Obj = -7.7401;\n",
      "Iter 2;\t Fid = 0.50;\t Obj = -85.9927;\n",
      "Iter 3;\t Fid = 0.50;\t Obj = -10.4996;\n",
      "Iter 4;\t Fid = 0.50;\t Obj = -90.1834;\n",
      "Iter 5;\t Fid = 0.50;\t Obj = -15.9023;\n",
      "Iter 6;\t Fid = 0.50;\t Obj = -121.0723;\n",
      "Iter 7;\t Fid = 0.50;\t Obj = -96.2876;\n",
      "Iter 8;\t Fid = 0.50;\t Obj = -8.3718;\n",
      "Iter 9;\t Fid = 0.50;\t Obj = -11.9013;\n",
      "Iter 10;\t Fid = 0.50;\t Obj = -15.0995;\n",
      "Iter 11;\t Fid = 0.50;\t Obj = -7.6864;\n",
      "Iter 12;\t Fid = 0.50;\t Obj = -6.4228;\n",
      "Iter 13;\t Fid = 0.50;\t Obj = -51.4467;\n",
      "Iter 14;\t Fid = 0.50;\t Obj = -22.0845;\n",
      "Iter 15;\t Fid = 0.50;\t Obj = -5.9313;\n",
      "Iter 16;\t Fid = 0.50;\t Obj = -130.6765;\n",
      "Iter 17;\t Fid = 0.50;\t Obj = -181.9663;\n",
      "Iter 18;\t Fid = 0.50;\t Obj = -88.9639;\n",
      "Iter 19;\t Fid = 0.50;\t Obj = -21.3274;\n",
      "Iter 20;\t Fid = 0.50;\t Obj = -8.2452;\n",
      "Iter 21;\t Fid = 0.50;\t Obj = -14.1170;\n",
      "Iter 22;\t Fid = 0.50;\t Obj = -8.3733;\n",
      "Iter 23;\t Fid = 0.50;\t Obj = -33.3266;\n",
      "Iter 24;\t Fid = 0.50;\t Obj = -22.8821;\n",
      "Iter 25;\t Fid = 0.50;\t Obj = -1.2653;\n",
      "Iter 26;\t Fid = 0.50;\t Obj = -6.2855;\n",
      "Iter 27;\t Fid = 0.50;\t Obj = -129.2954;\n",
      "Iter 28;\t Fid = 0.50;\t Obj = -27.6052;\n",
      "Iter 29;\t Fid = 0.50;\t Obj = -41.9348;\n",
      "Iter 30;\t Fid = 0.50;\t Obj = -22.8847;\n",
      "Iter 31;\t Fid = 0.50;\t Obj = -2.2557;\n",
      "Iter 32;\t Fid = 0.50;\t Obj = -13.0811;\n",
      "Iter 33;\t Fid = 0.50;\t Obj = -9.5395;\n",
      "Iter 34;\t Fid = 0.50;\t Obj = -13.6012;\n",
      "Iter 35;\t Fid = 0.50;\t Obj = -85.0653;\n",
      "Iter 36;\t Fid = 0.50;\t Obj = -79.1870;\n",
      "Iter 37;\t Fid = 0.50;\t Obj = -7.1699;\n",
      "Iter 38;\t Fid = 0.50;\t Obj = -8.6771;\n",
      "Iter 39;\t Fid = 1.00;\t Obj = -1.5034;\n",
      "Iter 40;\t Fid = 0.50;\t Obj = -122.4489;\n",
      "Iter 41;\t Fid = 0.50;\t Obj = -24.0085;\n",
      "Iter 42;\t Fid = 0.50;\t Obj = -2.1577;\n",
      "Iter 43;\t Fid = 1.00;\t Obj = -4.0370;\n",
      "Iter 44;\t Fid = 0.50;\t Obj = -13.9348;\n",
      "Iter 45;\t Fid = 1.00;\t Obj = -5.6706;\n",
      "Iter 46;\t Fid = 1.00;\t Obj = -3.8358;\n",
      "Iter 47;\t Fid = 1.00;\t Obj = -1.0238;\n",
      "Iter 48;\t Fid = 0.50;\t Obj = -289.3283;\n",
      "Iter 49;\t Fid = 1.00;\t Obj = -3.0379;\n"
     ]
    }
   ],
   "source": [
    "candidate_set = torch.rand(\n",
    "    1000, problem.bounds.size(1), device=problem.bounds.device, dtype=problem.bounds.dtype\n",
    ")\n",
    "candidate_set = problem.bounds[0] + (problem.bounds[1] - problem.bounds[0]) * candidate_set\n",
    "\n",
    "cumulative_cost = 0.0\n",
    "\n",
    "with botorch.settings.validate_input_scaling(False):\n",
    "    for it in range(N_ITER):\n",
    "        mll, model = initialize_mes_model(train_x_mes, train_obj_mes, data_fidelity=2)\n",
    "        fit_gpytorch_mll(mll)\n",
    "        acqf = qMultiFidelityMaxValueEntropy(\n",
    "            model, candidate_set, cost_aware_utility=cost_aware_utility\n",
    "        )\n",
    "        new_x, new_obj, cost = optimize_mes_and_get_observation(acqf,\n",
    "                                                                fixed_features_list=[{2: fid} for fid in fidelities])\n",
    "        train_x_mes = torch.cat([train_x_mes, new_x])\n",
    "        train_obj_mes = torch.cat([train_obj_mes, new_obj])\n",
    "        cumulative_cost += cost\n",
    "        print(\n",
    "            f\"Iter {it};\"\n",
    "            f\"\\t Fid = {new_x[0].tolist()[-1]:.2f};\"\n",
    "            f\"\\t Obj = {new_obj[0][0].tolist():.4f};\"\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd44be2238fc0110",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Plot results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "5c5cc1ef1808ad1f",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T08:03:49.194444700Z",
     "start_time": "2024-02-08T08:03:49.188327300Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "mapping_fid = dict(zip(range(fidelities.shape[0]), fidelities.tolist()))\n",
    "cost_AGP = torch.cumsum(torch.tensor([mapping_fid[int(source)] for source in train_x[:, -1].tolist()]), dim=0)\n",
    "cost_MES = torch.cumsum(train_x_mes[:, -1], dim=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "4a7f7020f195a31f",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T08:03:49.200521400Z",
     "start_time": "2024-02-08T08:03:49.194444700Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "train_obj[torch.where(train_x[:, -1] != fidelities.shape[0] - 1)] = train_obj.min()\n",
    "best_seen_AGP = torch.cummax(train_obj, dim=0)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f696548b9b3f50a5",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T08:03:49.206250200Z",
     "start_time": "2024-02-08T08:03:49.200521400Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "train_obj_mes[torch.where(train_x_mes[:, -1] != 1)[0]] = train_obj_mes.min()\n",
    "best_seen_MES = torch.cummax(train_obj_mes, dim=0)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "e3604083ed32e0eb",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-08T08:03:49.468051600Z",
     "start_time": "2024-02-08T08:03:49.206250200Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwwAAAJECAYAAAC7A6POAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAB7CAAAewgFu0HU+AAB19UlEQVR4nO3dd3wUdf7H8fem91BC770LoakUqaKCCOhhl6qnnqCeKArngXo/+2EXFUSwnCJ4CiIWFAEBUXqTJlUgCSGhhPRsdn5/cFmzu9kUyO7OJq/n48GD2ZnvzHyWMWY/+/kWi2EYhgAAAACgCAG+DgAAAACAeZEwAAAAAHCLhAEAAACAWyQMAAAAANwiYQAAAADgFgkDAAAAALdIGAAAAAC4RcIAAAAAwC0SBgAAAABukTAAAAAAcIuEAQAAAIBbJAwAAAAA3CJhAAAAAOAWCQMAAAAAt0gYAAAAALhFwgAAAADALRIGAAAAAG6RMAAAAABwi4QBAAAAgFskDAAAeFjfvn1lsVhksVi0cuVKX4cDAGVCwgAAKJPCH36L+hMQEKDo6Gg1adJEw4cP1+zZs3Xu3Dlfhw0AuEAkDACAcmUYhtLT03X48GEtXrxYf/3rX9WiRQt9+eWXvg4NAHABgnwdAADAf3Xr1k3du3d32Gez2XTmzBlt27ZNu3btkiSdOHFC119/vRYtWqRrr73WF6ECAC4QCQMA4IINHjxYTzzxhNvja9eu1c0336xjx44pPz9f99xzjw4dOqTg4GDvBWkCjFsA4M/okgQA8JiePXtq4cKF9tfHjx/nwzMA+BkSBgCAR1122WVq0qSJ/XVBNyUAgH8gYQAAeFydOnXs2xkZGS7HDx8+bJ9lqXHjxvb9a9as0Z133qnWrVsrNjZWFotFDz74oMO5NptNq1ev1rRp0zRo0CA1bNhQERERCg0NVZ06ddS/f389/fTTSklJKVWshWd8KrB37149+OCDatOmjaKiohQTE6OOHTtqypQppbpuaaZVHTNmjL3NvHnzJEmZmZmaOXOmevXqpVq1aik0NFQNGjTQLbfcorVr15bq/QDAxWIMAwDA45KSkuzbtWvXLrF9bm6u7r//fr3zzjvFtsvLy1OTJk10/Phxt/dNSkrSihUr9Oyzz+rtt9/W7bffXqbY3377bT344IPKyclx2L99+3Zt375ds2fP1rfffquuXbuW6bol2bVrl/7yl79o9+7dDvuPHTum+fPna/78+Zo2bZqefPLJcr0vADgjYQAAeNTGjRt18OBB++vevXuXeM7f//53e7LQoUMHdezYUcHBwdq3b58CAv4sjufn59uThaioKLVr105NmzZVTEyM8vLydOzYMf3yyy9KS0tTRkaG7rjjDgUHB+umm24qVezz5s3TvffeK0lq1aqVunbtqvDwcO3Zs0dr166VYRhKTU3Vddddp927dys2NrbU/y7FSUhI0MCBA5WYmKgqVaqod+/eql27tlJSUvTjjz/q7NmzkqSnnnpKbdu2LfX7AYALYgAAUAZ9+vQxJBmSjOnTpxfbdv369Ubjxo3t7UeMGFFku0OHDtnbBAYGGpKMBg0aGD/99JNL2+zsbPt2Tk6OMXbsWGPFihVGbm5ukdfOzs42XnjhBSMoKMiQZFSpUsU4d+6c25gL4pBkhIaGGjVq1DC++eYbl3arVq0yYmJi7G2ffPJJt9cs/G+2YsWKItuMHj3a4b6SjEcffdTIyMhwaJeammr079/f3rZp06aGzWZze28AuFhUGAAAF+zrr7926cNvs9l09uxZbd++XTt37rTvHzFihD766KMSr5mfn6+IiAj98MMPatmypcvx0NBQ+3ZISIjee++9Yq8XGhqqRx55RDabTY899pjOnDmjDz/80F45KMkPP/ygSy65xGX/FVdcoWeeeUYTJkyQJH3yySeaNm1aqa5ZkpycHE2ZMkXPPPOMy7Fq1arp448/VrNmzZSRkaGDBw9q/fr1uvTSS8vl3gDgjIQBAHDBNmzYoA0bNhTbpk6dOpo5c6aGDx9e6utOmDChyGThYowdO1aPPfaYpPNJQGkShr/+9a9FJgsFRo0apQcffFBWq1V79+5VWlqaYmJiLjrWGjVqFJt81KpVS0OGDNGCBQskiYQBgEeRMAAAPCoxMVE33HCDbr31Vr322muqWrVqiefcfPPNZb6PzWbTpk2btHXrVh07dkxpaWnKy8srsu3WrVtLdc2RI0cWezw6OlrNmjXT3r17ZRiGjhw5og4dOpQ1dBdDhw5VWFhYsW3i4+PtCcPhw4cv+p4A4A4JAwDggk2fPr3IlZ4zMjJ0+PBhffPNN3rhhRd08uRJffTRR9qyZYtWr15dbNIQHBxcpg/dVqtVr732ml5++WUdO3asVOeUdorV0sRRvXp1+3ZaWlqprmvW+wJAUViHAQBQ7iIjI9WuXTs9/PDD2rJli+rVqydJ+u233/TQQw8Ve27VqlUVFFS677NycnI0ZMgQTZo0qdTJgiSdO3euVO1KM+tRcHCwfdtdRaOsfHVfACgKCQMAwKPq1aun6dOn219/9NFHDusyOAsPDy/1tZ988kktW7ZM0vkF12666SYtWLBAu3fv1tmzZ5WbmyvDMOx/ChTeLk7hxdu8yVf3BYCi0CUJAOBxV111lX3barVq1apVF712QE5Ojl5//XX763nz5mnUqFFu25e2qgAAcESFAQDgcXXq1HF4feTIkYu+5vr165Weni5JateuXbHJQnndEwAqIxIGAIDHZWZmOrwuvFrzhUpISLBvl2aQ8E8//XTR9wSAyoiEAQDgcZs3b3Z4XTAI+mIUTjqcExJnNptNs2bNuuh7AkBlRMIAAPC4l19+2b5tsVjUv3//i75m06ZN7durVq3S2bNn3bZ98cUXtW3btou+JwBURiQMAACPOXPmjO6++24tWbLEvu/WW29VrVq1Lvra8fHx9krF2bNnNXLkSIduStL5gdHTpk3TY489psjIyIu+JwBURsySBAC4YF9//XWRi6BlZmbq8OHD+uWXX5SVlWXf37JlS7300kvlcu+AgAD961//0rhx4yRJ33//vVq2bKkePXqoUaNGSk1N1cqVK3X69GlJ0qxZs3TbbbeVy70BoDIhYQAAXLANGzZow4YNpWp73XXX6Z133lHNmjXL7f5jx47V/v379cwzz0g6v8L0999/79AmLCxMr7zyim699VYSBgC4ACQMAIByFxoaqtjYWDVv3lyXXXaZbr31VnXp0sUj93r66ad1zTXX6I033tCaNWt08uRJRUdHq379+rr66qs1fvx4tWjRwiP3BoDKwGKUdrlLAAAAAJUOg54BAAAAuEXCAAAAAMAtEgYAAAAAbpEwAAAAAHCLhAEAAACAWyQMAAAAANwiYQAAAADgFgkDAAAAALdIGAAAAAC4RcIAAAAAwC0SBgAAAABukTAAAAAAcIuEAQAAAIBbJAwAAAAA3ArydQAwr+zsbO3YsUOSVKNGDQUF8Z8LAACAWVmtVp08eVKS1KFDB4WFhZXLdfkECLd27Nih7t27+zoMAAAAlNH69evVrVu3crkWXZIAAAAAuEWFAW7VqFHDvr1+/XrVqVPHh9EAAACgOImJifbeIYU/x10sEga4VXjMQp06dVS/fn0fRgMAAIDSKs+xp3RJAgAAAOAWCYOfOHLkiCZNmqTWrVsrMjJS1apVU7du3fTiiy8qMzPT1+EBAACggqJLkh9YsmSJbr/9dqWlpdn3ZWZmauPGjdq4caPeffddLV26VM2bN/dhlAAAAKiIqDCY3JYtW3TTTTcpLS1NUVFRevrpp/Xzzz9r+fLluuuuuyRJ+/bt05AhQ3Tu3DkfRwsAAICKhgqDyT3wwAPKyspSUFCQli1bpssvv9x+rH///mrRooUmT56sffv2acaMGXriiSd8FywAAAAqHCoMJrZ+/XqtXr1akjR+/HiHZKHApEmT1KZNG0nSq6++qry8PK/GCAAAgIqNhMHEFi1aZN8eO3ZskW0CAgI0atQoSdKZM2e0YsUKb4QGAACASoKEwcTWrFkjSYqMjFSXLl3ctuvTp499e+3atR6PCwAAAJUHCYOJ7d69W5LUvHnzYhffaN26tcs5AAAAQHlg0LNJZWdnKyUlRZJKXGG5atWqioyMVEZGho4ePVrqexw7dqzY44mJiaW+FgAAAComEgaTKjxFalRUVIntCxKG9PT0Ut+jQYMGFxQbAAAAKg+6JJlUdna2fTskJKTE9qGhoZKkrKwsj8UEAACAyocKg0mFhYXZt3Nzc0tsn5OTI0kKDw8v9T1K6r6UmJio7t27l/p6AAAAqHhIGEwqOjravl2abkYZGRmSStd9qUBJYyMAAOZkGIbybYasNkN5+TZZ8w3l2f73d75NefmGJMPjcVhthjJz85WZk6+MXKsyc63KyMm3/52Vl6+MHKsyc//822qzeTwuwIxa1IzWv4a393UYF4SEwaTCwsJUvXp1paamljg4+fTp0/aEgXEJAHDx8m2GTmXkKjUjR6npuUpJz1FKeq5S03OUkv6/fRnnX2fn5Xs8Hpshe2JgtRUkBAD8SY7Vf5NlEgYTa9u2rVavXq39+/fLarW6nVp1z5499u2CVZ8BoKzOZObq5e/3aU/SORmGZMj439+SzfhzW4YhW+HjFeiza16+TacycnUqM7dCvS8AuBgkDCbWq1cvrV69WhkZGdq0aZMuvfTSItutWrXKvt2zZ09vhQegArHZDN31wUZtOHza16EAAEyGhMHEhg8frmeffVaSNHfu3CITBpvNpg8++ECSVKVKFfXr18+rMQKoGD7bdIxkARckIiRQESFBigz9398hgQoPCVRkSJAiQp3+DglUcCATNKJyqhEd6usQLhgJg4l1795dvXv31urVqzVnzhyNHj1al19+uUObGTNm2Fd3fuCBBxQcHOyLUAH4sbOZeXr+2z0lN6ykqkYEq3pUqOKiQs7/HRmiuKhQVY8KVfWoEEWFBsni6SAsUnBggIICLOf/DrQoKCBAwYEWBQUGKDjg/N9BgRYFB5z/O9Di8ahksUgWL9wHgG+RMJjcq6++qp49eyorK0uDBg3S1KlT1a9fP2VlZWn+/PmaNWuWJKlly5aaNGmSj6MF4I9e+n6vUjMcp2++v39z1YwJO/+BUJb//S0F/G/DovMfFAMs5z80VhQBFouqRYaoemSo4qJDVC0iREFm/0bcZpOOrJV+/07KPlu+1w6LlQb9X/leE4DfIWEwufj4eH366ae6/fbblZaWpqlTp7q0admypZYuXeowFSsAlMZvCWf14S9HHPYNaltLDw1q5aOIUGqnD0tbP5G2fSyd+cMz94iuS8IAgITBHwwdOlTbt2/Xq6++qqVLl+rYsWMKCQlR8+bNNXLkSE2YMEERERG+DhOAnzEMQ9MX/yZbodmAQoMC9M9r2/ouKBQvJ13a/aW05T/SkTW+jgZAJUHC4CcaNWqkl156SS+99JKvQwFQQXy++bg2HnEc6Hxfv+ZqUI0vIEzFZpP++Fna+rH02yIpL8PXEQGoZEgYAKASSsvO07PfOA50blgtQn+9oqmPIoKL04elbfPPJwpnjpTYXDXaSE16S5bA8oshLKb8rgXAb5EwAEAl9PL3+5SSnuOwb/rQtgoLLscPmxfjj1+kH/9PSjvu60h8w7CdTxhKElZF6jBS6nSrVDe+Yo1AB2AaJAwAUMnsSUrTB+scv7Ee0LqmBrSp5aOInOTnSR/fJGWf8XUk5mQJkJpfeT5JaHWNFOS/c7sD8A8kDABQiRiGoWmLf1N+oZHOIUEBmj60nQ+jcnLqIMlCUeJaSfG3SZfcJEXX9nU0ACoREgYAqES+3Jag9YdOOey7p08zNaxuooHOeZm+jsA8wmILdTnqTJcjAD5BwgAAlcS57Dw9vXS3w776VcP1t77NfBSRG3nZjq+DI6XrZ/kmFl8Ki5Hqd5eCw3wdCYBKjoQBACqJ15b/ruRzjgOd/3mtiQY6F7BmOb4OjZLaXOubWAAAMvl69wCA8vD7iXOau/aww74+LWtoUFuTDHQuzLnCEMQ37ADgSyQMAFDBFQx0thYe6BwYoCeuayeLGfvEO1cYgsN9EwcAQBIJAwBUeEt3JGrdwVSHfXdd0URN4iJ9FFEJXCoMTBsKAL5EwgAAFVhGjlX/95XjQOe6sWG6r19zH0VUCs4VhiAqDADgSyQMAFCBvf7jfiWlOX5j/89r2yoixMRzXlgdB2YzSxAA+BYJAwBUUAdOpmvOmoMO+3o1j9PV7U2+6FceFQYAMBMTf8UEACiwOzFN3+86oTOZeaU+59dDqcrL/3Ogc3CgxbwDnQuzOq/DQIUBAHyJhAEATCo7L19f70jUf379Q5uOnL7o643r1UTNa0aVQ2QeRoUBAEyFhAEATOZQSoY+/vWIFm46VqaKQnFqx4Tp/v4tyuVaHkeFAQBMhYQBAEwgL9+mH3ad0H9+/UNr9qeU67UtFumJ69opMtRP/pfvMq0qFQYA8CU/+e0BABVTwpkszV//h+ZvOKrkcznFtq0ZHareLWooKKD0YxBCgwN0Zdta6t2ixsWG6j0uC7dRYQAAXyJhAAAfWLs/RXPXHtKPe5JVaAHmIvVqHqfbL2uoAW1qKTiwEkxuR4UBAEyFhAEAvGzu2kN6csmuYttUjQjWyK4NdEv3huZdkdlTqDAAgKmQMACAFxmGobdXHXB7vGujqrrtsoa6pn0dhQUHejEyE3GpMJAwAIAvkTAAgBcdO52lE2mOYxWiQoM0Ir6ebr20odrUifFRZCbiUmGgSxIA+BIJAwB40eY/HNdTqBYZotWT+/nPDEbeQIUBAEylEoyeAwDzcF6ArXPDqiQLzpwrDCQMAOBTJAwA4EXOCUOXRlV9FImJOVcYGPQMAD5FwgAAXpKRY9XuxDSHfSQMRXCpMDCGAQB8iYQBALxk29EzDmsuBAVYdEn9WN8FZFZWpwXsqDAAgE+RMACAlzh3R2pXL7byTp3qjmFIVhZuAwAzIWEAAC/Z9IfzgOcqvgnEzJyTBYkKAwD4GAkDAHiBzWZoMwOeS5aX5bqPCgMA+BQJAwB4wYGT6UrLtjrsI2EoAhUGADAdEgYA8ALn8Qt1Y8NUJ5Zvzl1QYQAA0yFhAAAvcFmwjepC0VwqDBYpKNQnoQAAziNhAAAvcB7wTHckN5wXbQsKkywW38QCAJBEwgAAHnc6I1cHT2Y47CNhcMN50TbGLwCAz5EwAICHbTnqWF0ICw5QmzoxPorG5FwqDIxfAABfI2EAAA9zHr/QsX4VBQfyv98iUWEAANPhNxYAeJhzwkB3pGJQYQAA0yFhAAAPysu3advRsw77SBiK4VxhYIYkAPA5EgYA8KA9ieeUlZfvsC++IQmDW84VhmAqDADgayQMAOBBm46ccnjdtEakqkWG+CgaP+C8DkMQYxgAwNdIGADAgzb9ccbhdReqC8VzThioMACAz5EwAIAHbWbAc9nkOY9hoMIAAL5GwgAAHpJ4NkvHzzh+AO5MwlA8lwoDCQMA+BoJAwB4yOYjZxxeR4cFqXmNKN8E4y9cKgx0SQIAXyNhAAAPcV5/oXPDqgoIsPgoGj9BhQEATIeEAQA8ZNMfjF8oMyoMAGA6JAwA4AHZefnalcCCbWVGhQEATIeEAQA8YMfxs8rLN+yvAyxSxwZVfBeQv6DCAACmQ8IAAB7gPH6hde0YRYUG+SgaP0KFAQBMh4QBADzAOWGgO1IpUWEAANMhYQCAcmYYBgu2XSgqDABgOiQMAFDOjqRmKjUj12EfCUMp5TklDKz0DAA+R8IAAOXMuTtSjehQ1a9K15pSsTp3SSJhAABfI2EAgHLmsv5Cw6qyWFiwrVScKwzBJFoA4GskDABQzhi/cBGcxzBQYQAAnyNhAIBylJadp70nzjns60zCUDq2fMmW57iPCgMA+BwJAwCUo61/nJHx53ptCgkMUPt6Mb4LyJ84T6kqUWEAABMgYQCAcuQ84Ll9vRiFBgX6KBo/49wdSaLCAAAmQMIAAOVos/OAZ7ojlR4VBgAwJRIGACgn+TZDW/8447CPhKEMqDAAgCmRMABAOfk9+ZzO5Vgd9nVuSMJQas4VBkugFBjsm1gAAHYkDABQTpzHLzSoFq6aMXSpKTXnCgPVBQAwBRIGACgnzglDF6oLZeNcYWD8AgCYAgkDAJQTFmy7SFQYAMCUSBgAoBykpOfocGqmwz4WbCsjKgwAYEokDABQDpyrC5EhgWpVK9pH0fgplwoDCQMAmAEJAwCUg01O6y90alhFQYH8L7ZMqDAAgCnx2wwAyoHL+AUGPJedc4WBhAEATIGEAQAuUq7Vpm3HzjrsY/zCBWDQMwCYEgkDAFyk3xLOKtdqc9gXT4Wh7PKoMACAGZEwAMBFcl5/oWWtKMWGs0JxmVmdxjBQYQAAUyBhAICLtPkP1l8oF1QYAMCUSBgA4CIYhuFSYaA70gWiwgAApkTCAAAXIeFstk6k5Tjso8JwgagwAIApkTAAwEVwri5UiQhW07hIH0Xj56gwAIApkTAAwEVY8/tJh9ddGlaVxWLxUTR+jgoDAJgSCYMHpKen66efftK///1v3XjjjWrSpIksFossFosaN25c5uvt3LlTd999t5o1a6bw8HDVqFFDvXv31ttvvy2r1Vr+bwBAqeTbDP2wO9lh3+XNqvsomgqAdRgAwJSCfB1ARTR06FCtXLmyXK41e/ZsTZgwQbm5ufZ92dnZWrNmjdasWaO5c+dq6dKliouLK5f7ASi9jYdP6VRGrsO+q9rV9lE0FUCeU5ckKgwAYApUGDzAMAz7drVq1TRo0CBFRUWV+Tpff/217rnnHuXm5qpWrVp67bXX9Ouvv+qbb77R9ddfL0lav369RowYofz8/HKLH0DpLNt1wuF1mzoxalAtwkfRVABUGADAlKgweMCtt96qu+++W926dVPz5s0lSY0bN1Z6enqpr5GXl6eJEyfKZrMpJiZGa9euVbNmzezHr776at13332aOXOm1qxZow8//FBjxowp77cCwA3DMPTdb0kO+65qV8tH0VQQVBgAwJSoMHjAX//6V91yyy32ZOFCfPHFFzp48KAkacqUKQ7JQoEXX3xRVatWtW8D8J7died07LTjB9xBbemOdFGcKwwkDABgCiQMJrVo0SL7trvKQUREhG688UZJ0q5du7Rv3z4vRAZAkpbtcqwuNKgWrjZ1on0UTQXhXGEIJmEAADMgYTCpNWvWSJJatWql2rXdf2vZp08f+/batWs9HheA8777zXH8wqC2tZlO9WK5VBgYwwAAZkDCYELp6ek6evSoJKl169bFti18fPfu3R6NC8B5R09landimsO+QW0Zv3BRDKOIQc9UGADADBj0bELHjh2zb9evX7/Ytg0aNLBvFyQZF3KfoiQmJpbpekBl4TzYuVpkiLo2ruajaCqI/DzJsDnuo8IAAKZAwmBC586ds2+XNB1rZGSkfbssszBJjskGgNJznk51YJuaCgygO9JFsWa57qPCAACmQJckE8rO/rMsHxISUmzb0NBQ+3ZWVhG/cAGUq9T0HG08fMphH4u1lYO8bNd9VBgAwBQqbYWhPAYnzp071yNrH4SF/fmtWuEVnouSk5Nj3w4PL9sv15K6MCUmJqp79+5luiZQ0S3fnSzbn2szKiIkUD2bs9L6RaPCAACmVWkTBjOLjv5zasaSuhllZGTYt8u6mnRJ4yMAuHIev9C3VQ2FBQf6KJoKhAoDAJhWpU0YymNGoTp16pRDJK7q1atn3y5pYHLhKgFjEgDPysixavX+FId9LNZWTpwrDIEhUgC9ZgHADCptwlDSdKW+FB0drQYNGujo0aPas2dPsW0LH2/Tpo2nQwMqtZ/2nVSu9c+ZfIICLOrXuqYPI6pAnCsMVBcAwDT4+sakevXqJUnau3evkpKS3LZbtWqVfbtnz54ejwuozJy7I13erLpiw4N9FE0F41xhYPwCAJgGCYNJDR8+3L49b968IttkZmZqwYIFkqS2bduqZcuWXogMqJzy8m1avifZYR+LtZUjlwoDCQMAmAUJg0mNGDFCTZs2lSQ9++yzOnDggEubRx55RKdPn7ZvA/CcXw6m6ly21WHflYxfKD8uFQa6JAGAWVTaMQyetH//fq1Zs8ZhX8FsR+np6S4Vg6uvvlq1azt+8AgODtbrr7+uoUOHKi0tTT179tTjjz+u7t276/Tp05o9e7b++9//SjrffemOO+7w3BsCoGW/OS7W1rFBFdWO5VvwckOFAQBMi4TBA9asWaOxY8cWeSw1NdXl2IoVK1wSBkkaPHiw3n77bU2YMEEnTpzQxIkTXdp0795dX3zxhQIDmdYR8BSbzdCyXY7jF65qR3ekcuVcYSBhAADToEuSyd11113atGmT7rrrLjVt2lRhYWGqXr26evXqpbfeektr165VXByLRgGetP34WZ1Iy3HYx3Sq5cy5wsCgZwAwDSoMHjBmzJhyXQG6ffv2mjVrVrldD0DZOM+O1KxGpJrXLNtCiSiBS4WBMQwAYBZUGACgBMucEoZB7agulDurYwWHCgMAmAcJAwAUY39yug6czHDYx3SqHpBHhQEAzIqEAQCK4TzYuVZMqDrWr+KbYCoyK2MYAMCsSBgAoBjO06le2baWAgIsPoqmAnOZVpUKAwCYBQkDALiRdDZbW4+ecdh3FeMXPMNl4TYqDABgFiQMAODG97sdqwvRYUG6tEl1H0VTwVFhAADTImEAADecZ0ca0LqmQoL436ZHUGEAANPiNx8AFOFsVp7WHUh12Md0qh7kUmEgYQAAsyBhAIAirNiTLKvNsL8OCQpQn5Y1fBhRBedSYaBLEgCYBQkDABTBeTrV3s3jFBka5KNoKgEqDABgWiQMAOAkOy9fK/eedNg3qB2LtXkUFQYAMC0SBgBwsnZ/ijJz8+2vAyzSwDYkDB7lUmEI9U0cAAAXJAwA4MR5sbaujaqpehQfYD3KucLAtKoAYBokDABQSL7N0A9O6y/QHckLnCsMTKsKAKbhsxF8aWlpOnfunPLz80ts27BhQy9EBADSpiOnlZqR67BvUFumU/Uow5Dycxz3UWEAANPwasLw/fffa+bMmVqzZo1OnTpVqnMsFousVquHIwNQUWXn5Wvn8bMqNENqseav/8Phdeva0WpYPcIDkcHOmu26jwoDAJiG1xKG+++/X2+++aYkyTBK+ZsbAC7C+kOnNHbuemXkllzJdOcqFmvzvLws131UGADANLySMHz88cd64403JElhYWEaPny4unTpomrVqikggGEUADzjX1/tuqhkQWL8gldQYQAAU/NKwvDOO+9Ikho0aKAff/xRzZo188ZtAVRiKek52nH87EVdo3XtaLWtE1NOEcEtKgwAYGpeSRi2b98ui8Wi6dOnkywA8IpfDqY6vLZYpJiw4FKf36FerP55bVtZLJbyDg3OXCoMFtZhAAAT8UrCkJeXJ0mKj4/3xu0AQD8fcEwY+rSsoXlju/soGhTLZdG2sPMZHgDAFLwygKBx48aSpPT0dG/cDgC0zilh6NGsuo8iQYmcF21j/AIAmIpXEobrr79ekrR8+XJv3A5AJZdwJkuHUjIc9vVoFuejaFAilwoD4xcAwEy8kjBMmjRJDRs21CuvvKI9e/Z445YAKjHn7kix4cEMXjYzKgwAYGpeSRhiY2P13XffqVatWurRo4dmzpyp06dPe+PWACqhnw+kOLy+vGl1BQTQJ960qDAAgKl5ZdBz06ZNJUmZmZk6c+aMJk6cqPvvv19xcXGKiCh+BVWLxaIDBw54I0wAFYBhGK7jF5ozfsHUqDAAgKl5JWE4fPiww2vDMGQYhpKTk0s8lykNAZTFoZQMJZ51/Maa8QsmV9QsSQAA0/BKwjB69Ghv3AYAXMYv1IwOVbMakT6KBqXiXGEgYQAAU/FKwjB37lxv3AYAipxOlUqlyTlXGIIZwwAAZuKVQc8A4A02m6F1B50TBrojmZ7zSs9UGADAVEgYAFQYe5LO6VRGrsM+Bjz7AeeEgUHPAGAqXumS5CwrK0ubNm1SUlKSMjMzNXz4cMXEMEc6gIvjPJ1qw2oRql+1+JnYYAJ5zmMY6JIEAGbi1YTh6NGjmjp1qhYuXKi8vDz7/q5du6pt27b213PmzNE777yj2NhYLVu2jP7HAEqlqPEL8ANUGADA1LzWJenXX39VfHy8Pv74Y+Xm5tqnVi3K0KFDtX37dv34449atmyZt0IE4Mes+Tb9euiUw77LSRj8AxUGADA1ryQMZ86c0bBhw3Tq1CnVrl1bM2fO1I4dO9y2r1mzpq655hpJ0tKlS70RIgA/t/34WaXnWB32MeDZT1BhAABT80qXpNdee03JycmKi4vTunXr1LBhwxLPGThwoBYvXqz169d7IUIA/s65O1LLWlGqER3qo2hQJlQYAMDUvFJhWLJkiSwWix566KFSJQuS1K5dO0nSgQMHPBkagArCecAz1QU/QoUBAEzNKwnD/v37JUlXXHFFqc+pWrWqJCktLc0jMQGoOLLz8rXx8GmHfYxf8CNUGADA1LySMGRnn//2KDg4uNTnZGRkSJLCw/nFAaB4m/84rRyrzf46wCJd1pSEwW9QYQAAU/NKwlCzZk1J0qFDh0p9ztatWyVJdevW9URIACoQ5/EL7evFKja89F9QwMfynFd65osiADATryQMl156qSTpm2++KVV7wzA0e/ZsWSwW9e7d25OhAagAfnZKGOiO5Geszl2SGKwOAGbilYThtttuk2EY+s9//mOvHBRn0qRJ2rZtmyRp9OjRHo4OgD9Lz7Fq29EzDvsY8OxnnCsMwVQYAMBMvJIwDBs2TP369ZPVatWAAQP01ltvKTk52X7carUqISFBCxcuVO/evfXqq6/KYrHo+uuvV48ePbwRIgA/teHQKVltfy4CGRxoUbfGVX0YEcrMpcLAGAYAMBOvrMMgSf/97381YMAAbdmyRRMmTNCECRNksVgkSfHx8Q5tDcPQZZddpnnz5nkrPAB+ynk61fgGVRUR4rX/teFi5Vslm+OCe1QYAMBcvFJhkKQqVapo3bp1mjJlimJiYmQYRpF/wsPDNXnyZK1cuVKRkZHeCg+An2L8gp9zniFJosIAACbj1a/hQkJC9PTTT2vq1KlatWqVNm7cqOTkZOXn56t69eqKj4/XwIEDFRsb682wAPip0xm52pXouFZLDxIG/1JUwkCFAQBMxSd1+8jISA0ePFiDBw/2xe0BVBC/HEyV8efwBYUFByi+IeMX/Irzom0SFQYAMBmvdUkCgPLm3B2pW+NqCgnif2t+hQoDAJieTyoMWVlZ2rRpk5KSkpSZmanhw4crJibGF6EA8GPOA56ZTtUPOVcYLIFSIIvuAYCZeDVhOHr0qKZOnaqFCxcqLy/Pvr9r165q27at/fWcOXP0zjvvKDY2VsuWLbPPpgQABU6kZevAyQyHfYxf8EPOFQaqCwBgOl6r3f/666+Kj4/Xxx9/rNzcXPusSEUZOnSotm/frh9//FHLli3zVogA/IhzdSE6LEjt6zFhgt9xrjAwfgEATMcrCcOZM2c0bNgwnTp1SrVr19bMmTO1Y8cOt+1r1qypa665RpK0dOlSb4QIwM/8vN9x/MJlTasrMIBqpN+hwgAApueVLkmvvfaakpOTFRcXp3Xr1qlhw4YlnjNw4EAtXrxY69ev90KEAPyJYRguA57pjuSnqDAAgOl5pcKwZMkSWSwWPfTQQ6VKFiSpXbt2kqQDBw54MjQAfujoqSwdP+P4QZMBz37KpcJAwgAAZuOVhGH//v2SpCuuuKLU51Sten4u9bS0tBJaAqhs1jqNX4iLClHLWlE+igYXxaXCQJckADAbryQM2dnnv0EKDi79VHkZGednPwkP55cHAEfO3ZEubxbHbGr+yrnCEBTqmzgAAG55JWGoWbOmJOnQoUOlPmfr1q2SpLp163oiJAB+yjAMrXNZf4HxC37LucLAoGcAMB2vJAyXXnqpJOmbb74pVXvDMDR79mxZLBb17t3bk6EB8DO/J6crJT3XYR8Jgx9zqTAwhgEAzMYrCcNtt90mwzD0n//8x145KM6kSZO0bds2SdLo0aM9HB0Af7J2v2N1oV6VcDWsFuGjaHDRmFYVAEzPKwnDsGHD1K9fP1mtVg0YMEBvvfWWkpOT7cetVqsSEhK0cOFC9e7dW6+++qosFouuv/569ejRwxshAvATRU2nyvgFP5ZHhQEAzM4r6zBI0n//+18NGDBAW7Zs0YQJEzRhwgT7L/n4+HiHtoZh6LLLLtO8efO8FR4AP5BvM/TLQaeEoTndkfyalTEMAGB2XqkwSFKVKlW0bt06TZkyRTExMTIMo8g/4eHhmjx5slauXKnIyEhvhQfAD/yWcFbnsq0O+1h/wc9RYQAA0/NahUGSQkJC9PTTT2vq1KlatWqVNm7cqOTkZOXn56t69eqKj4/XwIEDFRsb682wAPiJtfsdqwvNakSqVgwfMP0aFQYAMD2vJgwFIiMjNXjwYA0ePNgXtwfgp352mU6V6oLfo8IAAKbntS5JAHAxcq02bTh8ymEf06lWAMySBACm55MKQ1FOnDihr776SikpKWrSpImuvfZaRUQwVSKA87YePaPsPJv9tcUiXdaUhMHvOS/cRoUBAEzHKwnD7t27NX36dFksFr3zzjuqUqWKw/Evv/xSt956q7Ky/vzFUb9+fS1evFidOnXyRogATM55dqS2dWJUNTLER9Gg3FBhAADT80qXpEWLFumzzz5TQkKCS7KQnJys22+/XZmZmQ6zJR09elRDhw5Venq6N0IEYHJb/jjt8Lp7k2o+igTligoDAJieVxKG5cuXy2Kx6Nprr3U5NnPmTKWnpysoKEgvvfSStm3bphdeeEEBAQFKSEjQ7NmzvREiABMzDENbj55x2BffsKpvgkH5osIAAKbnlYThjz/+kOS6QJt0fkE3i8WiUaNG6cEHH1SHDh308MMPa/z48TIMQ19++aU3QgRgYkdSM3U6M89hX3yDKr4JBuWLCgMAmJ5XEobk5GRJUs2aNR32p6Sk6LfffpMk3XrrrQ7HrrvuOknSrl27vBAhADPbctSxO1JcVKjqV+Wb6ArBucJAwgAApuOVhKFgMHN2tuMvhjVr1kg6v6Bbr169HI7VqVNHknTmzBnPBwjA1Lb8ccbhdXzDKrJYLL4JBuXHMFwrDMEkDABgNl5JGKpVOz84saBrUoHly5dLkrp27aqQEMfZTqxWqyQpKirKCxECMDPn8Qud6I5UMeTnSTIc9wVROQIAs/FKwtCxY0dJ0scff2zfl5WVpYULF8pisah///4u5xw5ckSSVKtWLW+ECMCksvPytSshzWFffMMqvgkG5cua5bqPCgMAmI5XEoabb75ZhmFoyZIluvnmm/XGG29o0KBBSk5OlsVi0S233OJyzq+//ipJatSokTdCBGBSO4+fldX257fQARbpkvpVfBcQyk9etus+KgwAYDpeSRhGjRqlXr16yTAMLVy4UA888IB+/vlnSdLYsWPVunVrl3M+//xzWSwW9ejRwxshAjAp5/ELLWtFKyrUNIvU42JQYQAAv+CVhCEgIEDffPONHnroIdWvX19BQUFq0KCB/vnPf+qtt95yaf/VV1/p8OHDkqTBgwd7I0QAJuW6/kIVn8QBD6DCAAB+wWtf00VGRurf//63/v3vf5fYtmfPnjp06JAk/+ySdPjwYS1ZskQrV67U9u3bdfz4cdlsNsXFxalr1666+eab9Ze//EVBQaX759+5c6def/11/fDDD0pISFBUVJRat26t2267TXfeeWeprwP4I+cVnuMbsGBbheFcYQgMkQK88j0WAKAMTPlJs2rVqqpa1T8/FPzzn//U008/LcMwXI4dP35cx48f1+LFi/XSSy/ps88+U8OGDYu93uzZszVhwgTl5uba92VnZ2vNmjVas2aN5s6dq6VLlyouLq7c3wvgayfSspVw1vFbaCoMFYhzhYHqAgCYEl/llLPExEQZhqHIyEjdfvvtmjt3rtasWaONGzfqww8/VLdu3SRJGzZs0MCBA5Wenu72Wl9//bXuuece5ebmqlatWnrttdf066+/6ptvvtH1118vSVq/fr1GjBih/Px8r7w/wJucxy9EhwapWQ2mWq4wnCsMjF8AAFMyZYXBn1WvXl3PP/+87r33XkVHRzsc69Kli2655RbdeuutWrBggX7//Xe99NJLmjZtmst18vLyNHHiRNlsNsXExGjt2rVq1qyZ/fjVV1+t++67TzNnztSaNWv04YcfasyYMZ5+e4BXOa/w3LFBFQUEsGBbheFSYSBhAAAzosJQzp5//nlNnjzZJVkoEBgYqJkzZ9oXqvvss8+KbPfFF1/o4MGDkqQpU6Y4JAsFXnzxRXvXrRdffLE8wgdMpagVnlGBuFQY6JIEAGZEwuAD1atX1yWXXCJJOnDgQJFtFi1aZN92VzmIiIjQjTfeKEnatWuX9u3bV65xAr5kzbdpx7GzDvtIGCoYKgwA4BdIGHwkJydH0vmKQ1HWrFkjSWrVqpVq167t9jp9+vSxb69du7YcIwR8a++Jc8rKcxyb05EF2yoWKgwA4BdIGHwgOTlZu3fvliS1adPG5Xh6erqOHj0qSUUualdY4eMF1wQqAufuSI2qR6h6VKhvgoFnuFQYeL4AYEYMevaBF198UVarVZLsXYoKO3bsmH27fv36xV6rQYMG9u2CJKO0Ct+nKImJiWW6HlCeXMYvNKjikzjgQc4VBqZVBQBTImHwsl9//VWvvPKKpPPJwL333uvS5ty5c/btqKjip5CMjIy0bxc3RWtRCicbgNlsdZohqRMJQ8XjXGFgWlUAMCW6JHnRiRMn9Je//EVWq1UWi0Xvv/++IiIiXNplZ//5S7RgNiV3QkP/LOFnZWUV0xLwH2cz83TgZIbDvviG/rmYI4phZeE2APAHXqkwNGnSRAEBAfruu+/UvHnzUp3zxx9/qG/fvrJYLG5nEroYFsvFz+U+d+7cUq99cO7cOQ0ZMsTeDei5555T//79i2wbFvbnt2yFV3guSsHgaUkKDy/bL9uSujAlJiaqe/fuZbomUB62Hjvj8DokKEBt6sT4Jhh4jnPCQIUBAEzJKwnDkSNHZLFYSvzwW1heXp4OHz5cLh/sfS07O1vDhg3Tpk2bJEkPP/ywJk+e7LZ94TUcSupmlJHx57ewJXVfclbS+AjAV7b84dgdqUO9WIUEURCtcFwGPVNhAAAzqrRjGMpjRqE6deqU2MZqterGG2/UihUrJEl33nlniYus1atXz75d0sDkwlUCxiSgoth69IzDa8YvVFAu06pSYQAAMzJtwnD27PkFm4rq418eSpqutDzYbDbdcccdWrJkiSTppptu0jvvvFPiedHR0WrQoIGOHj2qPXv2FNu28PGipmgF/I1hGKzwXFlQYQAAv2DaGv9HH30kSWrUqJGPI7lwd999t+bPny9JGjp0qD766CMFBJTun7xXr16SpL179yopKcltu1WrVtm3e/bseRHRAuZwKCVDZ7PyHPYx4LmCosIAAH7BIxUGd4N5x44d6zANaFFycnJ08OBBJScny2KxaNCgQZ4I0eMeeughvfvuu5KkAQMGaOHChQoKKv0/9/Dhw/XJJ59IkubNm6fHHnvMpU1mZqYWLFggSWrbtq1atmxZDpEDvuVcXagRHaq6sXyQrJBcKgw8ZwAwI48kDCtXrpTFYpFhGPZ9hmFow4YNZbpO06ZNNWXKlPIOz+OeeOIJvfzyy5KkHj16aPHixQ7Tn5bGiBEj1LRpUx08eFDPPvusRo4cqWbNmjm0eeSRR3T69Gn7NlAROI9fiG9QpUJMfoAiuFQY6JIEAGbkkYThiiuucPgFv2rVKlksFnXp0qXYCoPFYlFYWJjq1KmjHj166Oabby6xImE2r7/+up588klJ5wcvv/DCCzp06FCx57Rq1UrBwcEO+4KDg/X6669r6NChSktLU8+ePfX444+re/fuOn36tGbPnq3//ve/ks53X7rjjjs884YAL9vitGAb3ZEqMCoMAOAXPFZhKKyg3/68efPUtm1bT9zSNAo+xEvS8ePH7WMRinPo0CE1btzYZf/gwYP19ttva8KECTpx4oQmTpzo0qZ79+764osvFBgYeFFxA2aQlZuv3YnnHPYx4LkCo8IAAH7BK7MkjRo1ShaLRVWr8k1hWd111126/PLL9dprr2n58uVKSEhQZGSk2rRpo9tuu0133nlnmcZGAGa2M+Gs8m1/dmUMsJxfgwEVFBUGAPALXvmkOW/ePG/cxhScqyvloX379po1a1a5XxcwG+cF21rVjlFkKAlxhUWFAQD8gql+Ex84cEApKSlq3LixatWq5etwAHgZ6y9UMi4VhrJNDgEA8A6vrMOQnJysmTNnaubMmfYF2Qrbv3+/unTpopYtW6pHjx6qV6+ebrjhBvsMQAAqB5eEgRWeKy6bTcrPcdzHwm0AYEpeSRg+//xzTZgwQa+++qpiYx37I+fk5Oiaa67R1q1bZRiGDMOQzWbTokWLNGzYMG+EB8AEEs9mKSnN8RtnKgwVmHOyILFwGwCYlFcShmXLlslisWjEiBEux+bNm6cDBw5Ikq677jq9+uqrGjp0qAzD0Nq1a/Xpp596I0QAPrbVqboQHRakpnFRvgkGnpeX5bqPCgMAmJJXEoa9e/dKki677DKXYx9//LGk86tDL1q0SBMnTtTixYs1cOBAGYah+fPneyNEAD62xWnBtk4NqigggAXbKixrtus+KgwAYEpeSRhOnjwpSapfv77D/qysLP3yyy+yWCz661//6nBs3LhxkqTNmzd7I0QAPuZcYWD8QgVHhQEA/IZXEoYzZ86cv1mA4+1++eUX5eXlyWKxaODAgQ7HmjRpIun8gGkAFVtevk3bj59x2McKzxWcS4XBwixJAGBSXkkYoqLO90NOSkpy2F+wZkHbtm1dFnULDg6WJBYlAyqBvUnnlJ1nc9jXiQpDxVbUom0WuqABgBl5JWFo3bq1JOnbb7912P/f//5XFotFffr0cTmnILlgPQag4nNesK1JXKSqRob4KBp4hcuibYxfAACz8srX90OGDNEvv/yiWbNmqU2bNurdu7fmzZunXbt2yWKx6Prrr3c5p2DsQr169bwRIgAfKmrAMyo4lwoD4xcAwKy8kjBMmDBBM2fOVGJioiZMmOBw7PLLL1e/fv1czlmyZIksFou6devmjRAB+JDLgGfWX6j4qDAAgN/wSpek2NhY/fDDD+rcubN9cTbDMNS7d28tWLDApf22bdu0YcMGSdKVV17pjRAB+MjpjFwdTMlw2BffgAHPFR4VBgDwG14bUdymTRtt3LhRhw4dUlJSkurUqaPGjRu7bT937lxJ59dnAFBxbT12xuF1aFCAWteJ9k0w8B4qDADgN7w+BVGTJk3sU6a607FjR3Xs2NFLEQHwJefuSB3qxSo40CvFT/gSFQYA8Bv8VgbgU84Dnhm/UEk4VxhYgwEATMvrFQabzaYVK1Zo3bp1SkpKUmZmpp5++mnVqVPH3iY3N1dWq1WBgYEKDeWXCFBR2WyGtjpNqcqCbZWEc4UhmAoDAJiVVxOGr776Svfff7+OHDnisP/hhx92SBjeffddTZw4UVFRUUpISFBkZKQ3wwTgJYdSM5SWbXXYx5SqlYRLhYExDABgVl7rkjR79mwNGzZMhw8flmEYql69ugzDKLLtnXfeqdjYWKWnp+uLL77wVogAvGyL0/iFWjGhqhPLB8dKwZrj+JpBzwBgWl5JGH7//Xfdd999ks7PerRr1y4lJye7bR8SEqIbbrhBhmFo2bJl3ggRgA84r/Ac36CqLBaLj6KBV+U5VxjokgQAZuWVhOHll1+W1WpVu3bt9PXXX6t169YlntO7d29J0pYtWzwdHgAfca4wdGLAc+VhdR7DQIUBAMzKKwnDjz/+KIvFogcffFAhISGlOqd58+aSpKNHj3oyNAA+kplr1d4T5xz2xTN+ofKgwgAAfsMrCcOxY8ckqUxrKxQMdM7MzPRITAB8a8exs8q3/TmOKTDAog71Y30YEbyKCgMA+A2vJAwFfZLL8uE/NTVVkhQbywcIoCJyXn+hde1oRYR4faZn+AoVBgDwG1757VyvXj39/vvvOnjwoH1sQknWrFkjSWratKknQwNQBMMwtDvxnNKy8zx2j5/2nXR4zXSqlQwVBgDwG15JGPr27at9+/bp/fff1+jRo0tsf/bsWb399tuyWCzq37+/FyIEUCA7L183zfpF25wqAJ7Ggm2VDBUGAPAbXumSdPfdd8tisWjVqlWaN29esW1TU1M1fPhwJSUlKSgoSPfcc483QgTwPx+uO+L1ZEGS4pkhqXKhwgAAfsMrCUN8fLweeOABGYah8ePH66abbtKCBQvsx3/++Wd9/PHHuu+++9S8eXP99NNPslgs+uc//6lGjRp5I0QA//Pfzce8fs8G1cLVpDorulcqeU4JAxUGADAtr40wnDFjhnJycvTWW2/ps88+02effWYfDH333Xfb2xWs/vzggw/q8ccf91Z4ACTtSkjTniTHqU5jw4MV4MG11BpWi9AT17VTgCdvAvOxOnVJosIAAKbltYTBYrHozTff1PDhw/Xcc89p1apVstlsLm0uv/xyPf7447rmmmu8FRqA//ncqbpQNzZMax7tz4d5lD8qDADgN7w+h+GVV16pK6+8UufOndOWLVuUnJys/Px8Va9eXZ06dVJcXJy3QwIgyZpv0+JtCQ77hsfXI1mAZzhXGIJCfRMHAKBEPpv0PDo6WldccYWvbg/AyZr9KTp5Lsdh3/Wd6/koGlRo+VbJZnXcF0yFAQDMyiuDngGY3+ebjzu87lg/Vs1rRvsoGlRoztUFSQpiDAMAmJVpEobTp0/r5MmT9kHPALznXHaevvstyWHf9Z3r+ygaVHjWHNd9VBgAwLQ8mjBYrVbt3LlTmzZt0smTJ12OZ2dna9q0aapfv77i4uJUu3ZtRUdH6y9/+Yt+++03T4YGoJBvdiQpx/rnJARBARYN7VjXhxGhQnNetE2iwgAAJuaRhMEwDE2bNk1xcXHq2LGjunfvrtq1a6tXr17asGGDJCk3N1dXXXWVnn76aSUmJsowDBmGoczMTH3xxRfq3r27li9f7onwADj5fIvj7Eh9W9VUtcgQH0WDCs950TaJCgMAmJhHBj2PHTtWH374oSQ5dDH6+eefdfXVV+vXX3/VzJkztXr1aklStWrV1KJFC1mtVu3atUtZWVnKysrSbbfdpr179yo2NtYTYQKQdOx0pn45eMph3w0MdoYnOVcYLIFSYLBvYgEAlKjcKwwrVqzQBx98IEkKDQ3VDTfcoIcfflgjR45UeHi4zpw5o5dfflnz5s1TcHCwZs2apZMnT2rdunXasGGDUlJS9PDDD0uSTp48qXnz5pV3iAAKWbTFcbBzTFiQ+rep6aNoUCk4VxioLgCAqZV7hWHu3LmSpJo1a+rHH39UmzZt7Mf27Nmj/v37a9asWbLZbHrkkUd05513OpwfHh6uF154QTt27NB3332npUuX6oEHHijvMAHofAXQeXakoR3rKjQo0EcRoVJwrjAwfgEATK3cKwy//vqrLBaL/v73vzskC5LUunVr/f3vf1d+fr4k6Y477nB7ndGjR0sSg58BD9p69IwOpmQ47GPtBXgcFQYA8CvlnjAkJJxfKfbyyy8v8njh/c2bN3d7nRYtWkiSTp065bYNgIvzhVN3pEbVI9S5YVUfRYNKgwoDAPiVck8YMjLOf1tZrVq1Io9XqVLFvh0aGur2OmFh53+B5Obmll9wAOxyrTZ9uS3BYd/18fVlsVh8FBEqDZcKAwkDAJiZx9ZhcPehgw8jgDms2JusM5l5DvtGxNMdCV7gUmGgSxIAmJlpVnoG4F2fb3Zce6F742pqWD3CR9GgUqHCAAB+hYQBqIROZ+Tqxz3JDvtGMNgZ3sIYBgDwKx5ZuE2SZs6cqZo1XedyT07+80PKU0895fb8wu0AlK+vdiQqL//PRRVDggI0uEMdH0aESsW5wkDCAACm5rGE4a233nJ7rGAcw5NPPump2wMohnN3pCvb1lJsOCvtwkucKwxMqwoApuaRhMEwjJIbAfCJgyfTteWPMw77bqA7ErzJmuP4mgoDAJhauScMK1asKO9LAihHzmsvVI8MUe8WNXwUDSolKxUGAPAn5Z4w9OnTp7wvCaCc2GyGPt/smDBc16muggOZ/wBelMcYBgDwJ3xKACqRDYdP6fgZx293b+hc30fRoNKiwgAAfoWEAahEnKsLLWtFqV3dGB9Fg0qLCgMA+BUSBqCSyM7L19IdiQ77ru9cn9XX4X0uC7dRYQAAMyNhACqJZbtOKD3Han9tsUjDOtX1YUSotFi4DQD8CgkDUEk4r73Qs1mc6sTyzS58gAoDAPgVEgagEkg+l63Vv6c47LuetRfgK1QYAMCvkDAAlcCXWxOUb/tzQcWIkEBd1a62DyNCpUaFAQD8CgkDUAk4z450dbvaigz1yELvQMmoMACAXyFhACq43Ylp2pWY5rDvetZegC9RYQAAv0LCAFRwX2xxrC7UjgnT5c2q+ygaVHqGUUSFIdQ3sQAASoWEAajAbDZDi7c6JgzD4+spMIC1F+Aj+bmSDMd9QVQYAMDMSBiACuzIqUydSMtx2MfsSPAp5+qCJAUzhgEAzIyEAajA9iY5jl2oHhmilrWifRQNIMma47qPCgMAmBoJA1CB7U1Kd3jdqjbJAnzMSoUBAPwNCQNQge094VhhIGGAz+Vlu+6jwgAApkbCAFRge5LOObxuTcIAX3OuMASGSAH8KgIAM+P/0kAFlZ2Xr8MpGQ77GL8An3OuMFBdAADTI2EAKqj9yemyOc1eScIAn3OuMDB+AQBMj4QBqKD2OnVHalgtQpGhQT6KBvgflwoDCQMAmB0JA1BB7T3hmDAw4Bmm4FJhoEsSAJgdCQNQQTHgGaZEhQEA/A4JA1BB7XNKGBi/AFOgwgAAfoeEAaiAzmbmKSnN8ZtcKgwwBSoMAOB3SBiACmhPkuOCbSGBAWocF+mjaIBCnCsMJAwAYHokDEAF5DzguVnNKAUH8uMOE3CuMDCtKgCYHp8ggArIecBzq1pRPooEcGJl4TYA8DckDEAF5DzguVXtGB9FAjhxThioMACA6ZEwABWMYRguXZIY8AzTcBn0TIUBAMyOhMEDli5dqieeeEJDhgxRmzZtFBcXp+DgYFWtWlVdunTRpEmTtHfv3lJf78iRI5o0aZJat26tyMhIVatWTd26ddOLL76ozMxMD74T+KOEs9k6l2112MeibTANl2lVqTAAgNkF+TqAisZqteraa68t8tiZM2e0efNmbd68Wa+//rqeeuopPfbYY8Veb8mSJbr99tuVlvbnrDeZmZnauHGjNm7cqHfffVdLly5V8+bNy/V9wH/tdZohKTosSHVi+VAGk6DCAAB+h4TBA2JjY9W3b19deumlatq0qerUqaOIiAglJCRo5cqVeu+993T27FlNmTJFVapU0T333FPkdbZs2aKbbrpJWVlZioqK0pQpU9SvXz9lZWVp/vz5mj17tvbt26chQ4Zo48aNio7mW2RIe5PSHV63qhUti8Xio2gAJ1QYAMDvkDCUs6CgIKWmpiowMLDI49ddd50mTpyoLl266PTp05o2bZruuuuuIts/8MADysrKUlBQkJYtW6bLL7/cfqx///5q0aKFJk+erH379mnGjBl64oknPPW24EecKwx0R4KpsHAbAPgdxjB4gLtkoUCTJk104403SpJOnjypPXv2uLRZv369Vq9eLUkaP368Q7JQYNKkSWrTpo0k6dVXX1VeXt7Fho4KwHlKVQY8w1RcKgx0SQIAsyNh8JHC3Yeys7Ndji9atMi+PXbs2CKvERAQoFGjRkk6Pz5ixYoV5Rsk/E5evk0HTjp1SWJKVZgJFQYA8DskDD6QlZWlxYsXSzr/ob9ly5YubdasWSNJioyMVJcuXdxeq0+fPvbttWvXlnOk8DeHUzKUl2847GtViwoDTIQKAwD4HRIGL8nLy9Mff/yh+fPnq0ePHvr9998lSePGjStysPLu3bslSc2bN1dQkPuhJq1bt3Y5B5WXc3ek2jFhio0I9lE0QBGoMACA32HQswcdPnxYTZo0cXv8qquu0owZM1z2Z2dnKyUlRZJUv379Yu9RtWpVRUZGKiMjQ0ePHi1TfMeOHSv2eGJiYpmuB9/b67LCM9UFmAwVBgDwOyQMPhAXF6c333xTN9xwQ5EDpM+d+/NDX1RUVInXK0gY0tPTS2xbWIMGDcrUHubHgGeYHhUGAPA7JAweVK9ePe3YsUPS+QXdjh8/rm+//VZz5szRPffcowMHDmjKlCku5xUeBB0SElLifUJDQyWdHxuBym3vCccpVVsyfgFmYrNJ+TmO+0gYAMD0Km3CUB4LWc2dO1djxoxxezw4OFjt27e3v+7UqZOGDBmiu+66S/369dPUqVP1+++/67333nM4Lyzsz1+gubm5JcaRk3P+F3B4eNlK+yV1YUpMTFT37t3LdE34TkaOVUdPOSaNdEmCqVhdZ4Rj4TYAML9KmzD40iWXXKL/+7//09/+9jfNnTtXN998swYNGmQ/XngQdGm6GWVkZEgqXfelwkoaHwH/su+EY3ekwACLmtcs238TgEcVlTAEMYYBAMyu0iYM5TGjUJ06dS743GHDhulvf/ubJOmzzz5zSBjCwsJUvXp1paamljgw+fTp0/aEgTEJlZvzgOfG1SMUFlz8IoKAV1FhAAC/VGkThsLTkfpCjRo17NtHjhxxOd62bVutXr1a+/fvl9VqdTu1auFVogtWfUbl5Dzgme5IMJ28IsZZUWEAANNjHQYfOX78uH27qK5EvXr1knS+u9GmTZvcXmfVqlX27Z49e5ZjhPA3zl2SWtVihWeYjEuFwSIFhfokFABA6ZEw+MjChQvt2x06dHA5Pnz4cPv23Llzi7yGzWbTBx98IEmqUqWK+vXrV75Bwq+wBgNMr6gpVcthAgoAgGeRMJSzRYsWlbjg2U8//aSnnnpKkhQUFKRbbrnFpU337t3Vu3dvSdKcOXO0bt06lzYzZsywj8V44IEHFBzMir6V1clzOUrNcJxRizUYYDoui7YxfgEA/EGlHcPgKYsWLdJNN92kIUOGaMCAAWrXrp2qVKminJwcHThwQEuWLNGCBQtks9kkSdOmTVOrVq2KvNarr76qnj17KisrS4MGDdLUqVPVr18/ZWVlaf78+Zo1a5YkqWXLlpo0aZLX3iPMx7m6EBYcoIbVInwUDeCGS4WB8QsA4A9IGDwgNzdXX3zxhb744gu3bcLDw/V///d/euihh9y2iY+P16effqrbb79daWlpmjp1qkubli1baunSpQ5TsaLy2ZPkumBbQABdPWAyVBgAwC+RMJSzF154QX369NFPP/2knTt36sSJE0pOTlZAQICqVaumdu3aqX///ho1alSppmUdOnSotm/frldffVVLly7VsWPHFBISoubNm2vkyJGaMGGCIiL4Jrmycx3wTAIJE6LCAAB+iYShnNWsWVNjx47V2LFjy+2ajRo10ksvvaSXXnqp3K6JioUBz/ALVBgAwC8x6BnwczaboX0nHFcEb12bKVVhQlQYAMAvkTAAfu6PU5nKyst32NeytuvaHoDPUWEAAL9EwgD4ub1O4xeqRYaoRhSLYcGEilqHAQBgeiQMgJ9zGb9QK1oWFsOCGTlXGEgYAMAvkDAAfo4Bz/AbzhUGuiQBgF8gYQD8nPMaDKzwDNOyMugZAPwRCQPgx7Lz8nU4NdNhX0sSBpiVc8JAhQEA/AIJA+DHDpxMV77NcNjXkkXbYFZ5zmMYqDAAgD8gYQD8mPP4hQbVwhUVynqMMCkqDADgl0gYAD/mOkMSC7bBxKgwAIBfImEA/NgelxmSWLANJkaFAQD8EgkD4Mf2nXBOGKgwwMSoMACAXyJhAPzU2cw8JZ51/MaWKVVhalQYAMAvMToS8FN7naoLwYEWNYmL9FE0QCk4L9xGhQHwmaysLKWlpSkjI0P5+fm+DgfFCAwMVGRkpGJiYhQe7pv/b5IwAH5qr9OCbc1qRCk4kKIhTMzq1CWJCgPgE2fPnlVCQoKvw0ApWa1W5eTk6NSpU6pbt65iY2O9HgMJA+CnnCsMreiOBLOjwgD4XFZWlkuyEBTEx0Ezs1qt9u2EhASFhoYqLMy7X7jwXwjgp1ymVCVhgNlRYQB8Li3tz+p0TEyMateurcDAQB9GhJLk5+crKSnJ/uzOnj3r9YSB/guAHzIMw2VKVQY8w9TyrZLN6rgviIQB8LaMjAz7NsmCfwgMDFTt2rXtrws/Q28hYQD8UOLZbJ3LdvzwxZSqMDXn6oJEwgD4QMEA56CgIJIFPxIYGGjvOuaLQeokDIAfch6/EB0apLqxfPiCiTmPX5CkYMYwAIA/IGEA/JDz+IWWtaNlsVh8FA1QCs5rMEhUGADAT5AwAH6IAc/wO0UlDFQYAMAvkDAAfogBz/A7eU5jGCyBUmCwb2IBAJQJCQPgZ/LybTqQnO6wr2UtEgaYnHOFgeoCABMYN26cLBaLqlevrpycnGLbbt26Vffcc4/atm2rmJgYhYSEqHbt2rryyis1Y8YMnTx50uUci8Xi8CcoKEh16tTR8OHD9dNPP3nqbZU71mEA/MyR1Azl5tsc9lFhgOk5VxgYvwDAx86dO6cFCxbIYrHo1KlTWrRokW666SaXdjabTZMnT9aMGTMUGBioK664QoMGDVJkZKSSk5O1bt06Pfzww5o+fbr27t2revXqOZxfvXp1TZgwQZKUnZ2trVu3avHixfryyy/16aefauTIkV55vxeDhAHwM87dkWrFhKpKRIiPogFKiQoDAJP59NNPlZGRoYceekivvPKK5syZU2TC8I9//EMzZsxQ586d9emnn6p58+YubTZv3qxHH31UWVmuU0jHxcXpiSeecNj37rvv6q677tLkyZP9ImGgSxLgZ1wHPLP+AvwAFQYAJjNnzhwFBQVp8uTJ6tevn5YvX64jR444tNm3b59efPFF1ahRQ99++22RyYIkde7cWd9//70aN25cqnuPGzdOkZGROnz4cJFdmcyGCgPgZ5wrDK1qRfkoEqAMXCoMJAyAmdhshk5n5vo6jFKrGhGigIALn058165d+uWXXzR48GDVqlVLo0aN0vLlyzV37lyHasD777+v/Px83X333apRo0aJ1y1YXK0s/GFadBIGwM/sO0GFAX7IpcJAlyTATE5n5qrL//3g6zBKbdPjA1U9KvSCz58zZ44k6Y477pAkXX/99frb3/6muXPnatq0aQoION8JZ926dZKkfv36XWTEjt5//31lZGSoSZMmiouLK9drewIJA+BHMnOt+uNUpsM+BjzDL1BhAGASeXl5+vDDDxUTE6Phw4dLkqKiojRixAh99NFH+uGHHzRo0CBJUlJSkiSpbt26LtdZuXKlVq5c6bCvb9++6tu3r8O+lJQUe9UiOztb27Zt07fffquAgAC9+OKL5frePIWEAfAj+06kyzD+fB1gkZrXpEsS/AAVBgAmsXjxYp08eVLjx49XWNifX16MGjVKH330kebMmWNPGIqzcuVKPfnkky77nROG1NRUe7vAwEDFxcVp2LBhmjRpknr37n1xb8ZLGPQM+JG9SWkOrxvHRSosONBH0QBlQIUBgEkUdEcaNWqUw/4BAwaoXr16Wrx4sU6dOiVJqlWrliQpISHB5TpPPPGEDMOQYRj65JNP3N6vVatW9nZWq1VJSUlatGiR3yQLEhUGwK+4DnimOxL8BLMkAaZWNSJEmx4f6OswSq3qBU4nfvToUS1btkyS1KdPH7ftPvroI91///3q0aOHVq5cqRUrVqh///4XdM+KgIQB8BPJadn6fPNxh32tGL8Af+FcYSBhAEwlIMByUYOI/cW8efNks9nUq1cvtWrVyuW41WrV+++/rzlz5uj+++/X6NGj9dxzz2nWrFl64IEH/GKAsieQMAB+wDAMTf1ip85m5Tns792i5CneAFNg4TYAPmYYhubOnSuLxaL3339fTZs2LbLdvn37tG7dOm3cuFFdu3bV5MmT9dxzz+maa67RJ598UuRaDGfOnPFw9L5FwgD4gS+3JeiH3Scc9l3Xsa66NKrqo4iAMsqjwgDAt3788UcdOnRIffr0cZssSNLYsWO1bt06zZkzR127dtXTTz+t3NxcvfTSS2rdurWuuOIKdezYUREREUpOTtb27du1fv16RUVFqVOnTt57Q17EoGfA5JLPZWv6l7857IuLCtET17XzUUTABbA6jWGgwgDAywoGO48ZM6bYdjfddJPCw8P1ySefKCsrSwEBAZoxY4Y2b96s8ePHKzExUe+++65efPFFLVmyRFFRUXrxxRd14MAB+zStFQ0VBsDEDMPQ41/s1JlMx65I/ze8vapFXtiAL8AnqDAA8LGPP/5YH3/8cYntYmJilJmZ6bI/Pj5e77zzTpnuaRSeC92PUWEATOzLbQlatsuxK9K1l9TR1e3r+Cgi4AIxhgEA/BYJA2BSJ8/luHRFqh4ZoifpigR/xLSqAOC3SBgAEzIMQ48v2uHSFelfw9tXimnvUAFRYQAAv0XCAJjQV9sT9d1vjl2RhlxSR4M70BUJfooKAwD4LRIGwGRS0nM0bfFOh33VIkP0FF2R4M+oMACA3yJhAExm2uKdOu3cFWkYXZHg56gwAIDfImEATGTp9kR9vSPJYd/gDrU15BK6IsHPUWEAAL9FwgCYREp6jv7p1BWpakSwnhrW3kcRAeXEMKgwAIAfI2EATGL64t90KiPXYd9Tw9orjq5I8Hf5uZKcFi8iYQAAv0HCAJjA0u2JWroj0WHf1e1q61q6IqEicK4uSFIwCQMA+AsSBsDHUouYFalqRLD+Nby9LBaLj6ICypHz+AVJCmIMAwD4CxIGwMemf/mbUp26Ij05rL1qRNMVCRVEUQkDFQYA8BskDIAPfbMjUV9td+yKdFW7WhpKVyRUJHlUGADAnwX5OgCgsrHm2/TT7yf12aZj+mFXssOxKnRFQkVkdRrDEBgiBfB9FQD4C/6PDXjJvhPn9MzXu3XZsz9q3LyN+npHknLzbQ5tnryunWpG01UDFYxzhYHqAgAfOnz4sCwWiywWi2rXri2r1Vpku927d9vbNW7c2L5/3rx59v3u/owZM8bhWhkZGXrmmWfUuXNnRUVFKTQ0VPXr11fv3r01ZcoUHThwwIPv+OJRYQA86Exmrr7clqDPNh3T9mNni207qG0tXdexrpciA7zIucLA+AUAJhAUFKQTJ07o66+/1nXXXedyfM6cOQoopho6YMAA9erVq8hjnTp1sm+fO3dOvXr10vbt29W8eXPdfvvtql69ulJSUrR+/Xo999xzatasmZo1a3bR78lTSBiAcubc5ci5iuAsJChAI7vU1z+vbUtXJFRMLhUGEgYAvtejRw9t27ZN7733nkvCYLVa9dFHH2ngwIFatWpVkecPHDhQjz32WIn3eeWVV7R9+3bdeeedmjVrlsvv+kOHDiknJ+fC34gXkDAARTAMQ19uS9D6Q6dkM4yST/ifXKuhn34/qZPnSv7B79Sgiv7Spb6GXlJXsRHBFxMuYG4uFQa6JAHwvfDwcN18882aM2eOkpOTVbNmTfuxr776SidOnNC4cePcJgyltW7dOknSfffdV+QXg02aNLmo63sDCQPgJC/fpqmf79DCTcfK/do1o0N1fef6+kuXempeM7rcrw+YEhUGwPxsNinrlK+jKL3wauUyecK4ceP0zjvv6MMPP9SkSZPs+9977z1Vq1ZNw4cPv+h7VK9eXZK0b98+h65K/oSEASgkI8eq+z7erJV7T5bbNUMCA3Rlu1r6S5f66t08TkGBzDWASoYKA2B+WaekF83bh97FIwekyLiLvkz37t3Vvn17zZ07154wJCUl6ZtvvtG9996r0FD3ayL98MMPys4uYtpoSTfffLNat24tSRo5cqQ++ugj3XnnnVq/fr0GDRqkLl262BMJf0DCAPxPSnqOxs3bUOLg5NLqaO9yVEdVIkLK5ZqAX6LCAMDExo0bp4ceeki//vqrLr30Ur3//vuyWq0aN25csectX75cy5cvL/JYp06d7AnDddddpxkzZmj69OmaMWOGZsyYIUlq1qyZrr76aj3wwANq0aJF+b6pckbCAEg6kpqhUe+t15HUTIf9YcEB+kuX+goqQ9kzLipEV7WrrRa16HIESKLCAMDUbr/9dj366KN67733dOmll2ru3LmKj48vsfvQs88+W6pBz5L00EMP6a677tK3336rn3/+WRs3btSvv/6qN998U3PmzNGnn35a5ExNZkHCgEpv+7EzGjt3g1Izch32V40I1ruju6lLo6o+igyoIKgwADCxGjVqaOjQoZo/f75GjhypvXv36vXXXy/3+0RHR2vkyJEaOXKkJOns2bOaOnWqZs6cqfHjx+v48eMKCTFnjwQSBlRqK/Ym677/bFZmbr7D/vpVw/X+uO5qViPKR5EBFYiVhAEwvfBq58cF+IvwauV6ufHjx+vzzz/XmDFjFBYWpttuu61cr1+U2NhYvfHGG1q6dKmOHDmiHTt2qEuXLh6/74UgYUCltXDjUT32+Q7l2xynTW1bJ0bzxnZTzRg+1ADlIo+F2wDTCwgol0HE/uqqq65SvXr1dPz4cd18882qWtU7vQssFosiIyO9cq+LQcKASscwDL25Yr/+vWyfy7FezeP01u2dFR3GughAubE6rUsSxBgGAOYSGBioRYsW6dixY+U+9ek777yjzp07q1u3bi7HFi1apN27d6tKlSpq3759ud63PJEwoFLJtxma/uVOffTLHy7Hhneqqxf+0lEhQUx7CpQrl0HPVBgAmE/Xrl3VtWvXUrcvblrV2rVr65577pEkffPNN7rnnnvUvHlz9ezZU3Xr1lVGRoa2bNmi1atXKyAgQDNnzix2CldfI2FApZGdl6/7P9miZbtOuBy7u09TPXpVawUEuK7ACOAiuQx6psIAwP8VN61qx44d7QnD888/r549e+r777/XTz/9pMTERElSvXr1NHr0aE2cONG0YxcKkDDANI7+vk0ph3Z45NqGIX2/K0k6ma5BhQsIFummrg00oLEh7fWjwV6APzl71PE1FQYAPtS4cWMZhlFyw/9xriKMGTNGY8aMKfX5rVq10iOPPKJHHnmk1OeYDQkDTOPYz5/q8kNveuz6nSWpqNnKtv3vDwDvYJYkAPArdNYGAHhXcISvIwAAlAEJAwDAu+qZu68uAMARXZJgGgHhVXXMUsej9wgJClC1yBAFMbgZ8L6wKlL3u6SarX0dCQCgDEgYYBqX3viIJP8dEAQAAFAR0SUJAAAAgFskDAAAAADcImEAAAAA4BYJAwAAQCURGBgoSbJarcrPz/dxNCit/Px8Wa1WSX8+Q28iYQAAAKgkIiMj7dtJSUkkDX4gPz9fSUlJ9teFn6G3MEsSAABAJRETE6NTp05JktLS0pSWlqagID4OmllBZaFAbGys12PgvxAAAIBKIjw8XHXr1lVCQoJ9n/MHUphX3bp1FRYW5vX7kjAAAABUIrGxsQoNDdXZs2eVkZFBtySTCwwMVGRkpGJjY32SLEgkDAAAAJVOWFiYzz58wv8w6BkAAACAWyQMAAAAANwiYQAAAADgFgkDAAAAALcY9Ay3Ck+zlpiY6MNIAAAAUJLCn9fKc7pcEga4dfLkSft29+7dfRgJAAAAyuLkyZNq3LhxuVyLLkkAAAAA3LIYhmH4OgiYU3Z2tnbs2CFJqlGjhumXjk9MTLRXQtavX686der4OCJ4C8++cuK5V148+8qJ514yq9Vq7yHSoUOHcltrw9yfAOFTYWFh6tatm6/DuCB16tRR/fr1fR0GfIBnXznx3Csvnn3lxHN3r7y6IRVGlyQAAAAAbpEwAAAAAHCLhAEAAACAWyQMAAAAANwiYQAAAADgFgkDAAAAALdIGAAAAAC4xcJtAAAAANyiwgAAAADALRIGAAAAAG6RMAAAAABwi4QBAAAAgFskDAAAAADcImEAAAAA4BYJAwAAAAC3SBgAAAAAuEXCAAAAAMAtEgYAAAAAbpEwwLSSk5P11Vdfadq0abrmmmsUFxcni8Uii8WiMWPGlPl633zzjUaMGKH69esrNDRU9evX14gRI/TNN9+Uf/C4YBs3btRTTz2lQYMG2Z9VVFSUWrZsqbFjx2rNmjVluh7P3T+kpaVp/vz5mjRpkvr06aPmzZsrNjZWISEhqlmzpvr27asXXnhBqamppbrezz//rNtvv12NGjVSWFiYateurauuukqffPKJh98JytOjjz5q//++xWLRypUrSzyHn3n/UfjZFvenb9++JV6L5+5hBmBSktz+GT16dKmvk5+fb4wfP77Y6915551Gfn6+594MSqV3797FPqeCP6NGjTJycnKKvRbP3b98//33pXr2cXFxxrffflvstaZPn24EBAS4vcaQIUOMrKwsL70zXKgtW7YYQUFBDs9uxYoVbtvzM+9/SvMzL8no06eP22vw3L2DhAGmVfiHvWHDhsagQYMuKGF47LHH7OfFx8cbn3zyibF+/Xrjk08+MeLj4+3HpkyZ4rk3g1Jp1qyZIcmoW7eu8cADDxifffaZsX79emPdunXGSy+9ZNSrV8/+vG655ZZir8Vz9y/ff/+90aBBA2PUqFHGq6++anz++efGunXrjLVr1xqffvqpMXLkSCMwMNCQZISEhBhbt24t8jpvv/22/dk2a9bMmDNnjrF+/Xpj0aJFRr9+/Ur93w98Kz8/3+jWrZshyahZs2apEgZ+5v1PwTO59957jR07drj9c/DgQbfX4Ll7BwkDTGvatGnGkiVLjKSkJMMwDOPQoUNlThj27t1r/4aqa9euRmZmpsPxjIwMo2vXroYkIygoyPj999/L+22gDIYMGWJ8+umnhtVqLfL4yZMnjZYtW9r/O1i1alWR7Xju/sfdMy/siy++sD/7ESNGuBxPTU01YmNj7V8ynDx50uUeQ4cOLdWHT/jWyy+/bEgyWrdubUyZMqXEZ8bPvH8qeK7Tp0+/oPN57t5DwgC/cSEJw7333ms/Z926dUW2Wbdunb3N3/72t3KMGJ6wZMkS+/OaOHFikW147hVXq1at7F2TnD3//PP2Z/rJJ58Uef7Ro0ftlYrBgwd7OlxcgCNHjhhRUVGGJGPlypXG9OnTS0wY+Jn3TxebMPDcvYdBz6iwDMPQ4sWLJUmtW7fWZZddVmS7yy67TK1atZIkLV68WIZheC1GlF2/fv3s2wcOHHA5znOv2KKjoyVJ2dnZLscWLVokSYqJidH1119f5Pn169fXwIEDJUnLly/XuXPnPBMoLth9992n9PR0jR49Wn369CmxPT/zlRPP3btIGFBhHTp0SAkJCZJU4i+dguPHjx/X4cOHPR0aLkJOTo59OzAw0OU4z73i2rt3r7Zu3Srp/AeEwnJzc7V+/XpJ0uWXX66QkBC31yl47jk5Odq4caNngsUFWbBggb766itVq1ZN//73v0t1Dj/zlRPP3btIGFBh7dq1y77t/OHCWeHju3fv9lhMuHirVq2yb7dp08blOM+9YsnMzNTvv/+ul156SX369JHVapUkPfjggw7t9u3bp/z8fEk8d3915swZPfDAA5Kk559/XnFxcaU6j595/7dw4UK1bdtWERERio6OVosWLTR69GitWLHC7Tk8d+8K8nUAgKccO3bMvl2/fv1i2zZo0MC+ffToUY/FhItjs9n03HPP2V/feOONLm147v5v3rx5Gjt2rNvjjz32mG699VaHfTx3/zd58mQlJSWpZ8+eGj9+fKnP49n7v8If/iVp//792r9/vz744AMNHz5c8+bNU2xsrEMbnrt3kTCgwircNzkqKqrYtpGRkfbt9PR0j8WEi/Pyyy/bu51cf/316tKli0sbnnvF1alTJ82aNUvdunVzOcZz92+rV6/Wu+++q6CgIL399tuyWCylPpdn778iIiJ03XXXacCAAWrdurWioqJ08uRJrVq1Sm+//bZSU1O1aNEiDRs2TN9//72Cg4Pt5/LcvYuEARVW4UGRxfVnlqTQ0FD7dlZWlsdiwoVbtWqVHnvsMUlSzZo19dZbbxXZjufu/4YPH66uXbtKOv9cDhw4oAULFuiLL77QLbfcoldeeUXXXnutwzk8d/+Vm5urv/71rzIMQ3//+9/Vvn37Mp3Ps/dfx48fV5UqVVz2X3nllZo4caKuueYabdmyRatWrdJbb72l+++/396G5+5djGFAhRUWFmbfzs3NLbZt4YG04eHhHosJF+a3337TiBEjZLVaFRYWpoULF6pmzZpFtuW5+78qVaqoffv2at++vbp166abb75Zn3/+uT744AMdPHhQw4YN07x58xzO4bn7r2eeeUZ79uxRw4YNNX369DKfz7P3X0UlCwVq1aqlzz77zF5VeP311x2O89y9i4QBFVbB9ItSySXIjIwM+3ZJpU1416FDhzRo0CCdPn1agYGBmj9/vq644gq37XnuFdcdd9yhkSNHymazacKECTp16pT9GM/dP+3Zs0fPPvuspPMfCAt3HSktnn3F1bRpU1155ZWSzo9rKJgVSeK5extdklBhFR4EVXhwVFEKD4IqPDgKvpWQkKCBAwcqISFBFotF7733noYNG1bsOTz3im3YsGFasGCBMjIy9O2339oHP/Pc/dPLL7+s3NxcNW3aVJmZmZo/f75Lm507d9q3f/zxRyUlJUmShg4dqsjISJ59Bde2bVt9/fXXks53Yapbt64kfua9jYQBFVbbtm3t23v27Cm2beHjRU3VCe9LSUnRlVdeqYMHD0o6/+3jqFGjSjyP516x1ahRw7595MgR+3bLli0VGBio/Px8nrsfKegqcvDgQd1yyy0ltv/Xv/5l3z506JAiIyP5ma/g3A2A57l7F12SUGE1adLE/k1E4bn7i/LTTz9JkurVq6fGjRt7OjSU4OzZs7rqqqvsU+0999xzuu+++0p1Ls+9Yjt+/Lh9u3DXgpCQEHXv3l2StG7dumL7NBf8dxEaGmofXA3/xc98xVZ4ytWC5yzx3L2NhAEVlsVisXdf2bNnj3755Zci2/3yyy/2bx+GDRtWpun8UP4yMzM1ZMgQbd68WZL0j3/8Q48++mipz+e5V2wLFy60b3fo0MHh2PDhwyVJaWlp+vzzz4s8/9ixY/rhhx8kSQMGDHDoBw3vmzdvngzDKPZP4YHQK1assO8v+ODHz3zFdejQIX3//feSpGbNmqlevXr2Yzx3LzMAP3Ho0CFDkiHJGD16dKnO2bt3rxEYGGhIMrp27WpkZmY6HM/MzDS6du1qSDKCgoKMffv2eSBylFZOTo4xaNAg+3N+4IEHLug6PHf/M3fuXCMrK6vYNi+99JL9v40mTZoYVqvV4XhqaqoRGxtrSDIaNWpkpKSkOBy3Wq3G0KFD7ddYsWJFeb8NeMD06dNLfGb8zPufL7/80sjLy3N7PCkpyYiPj7c/+xkzZri04bl7D2MYYFpr1qzR/v377a9TUlLs2/v373eZVnHMmDEu12jZsqUeeeQRPffcc9q4caN69uypRx99VM2aNdOBAwf0/PPPa8uWLZKkRx55RC1atPDIe0Hp3HLLLVq2bJkkqX///ho/frzDgEdnISEhatmypct+nrv/eeKJJzRp0iTdcMMN6tWrl5o1a6aoqCidO3dOO3bs0H/+8x+tXbtW0vnnPmvWLAUGBjpco1q1anr++ed1zz336MiRI7r00kv1j3/8Qx06dFBCQoJeeeUVrVixQtL5/9b69u3r7bcJD+Fn3v9MnDhReXl5uuGGG3T55ZercePGCg8PV0pKilauXKl33nnH/nu/V69eRXZL5bl7ka8zFsCd0aNH279ZKM0fd/Lz841x48YVe+748eON/Px8L747FKUsz1v/+xbZHZ67f2nUqFGpnnn9+vWNZcuWFXutadOmGRaLxe01Bg8eXGI1A+ZRmgqDYfAz729K+zN/ww03GKdPn3Z7HZ67d1gMwzAuKNMAPGzMmDF6//33S92+pP+Uv/76a82aNUsbNmxQSkqK4uLi1K1bN91999265pprLjZclIOy9i1t1KiRDh8+XGwbnrt/2Lt3r5YuXaq1a9dq//79OnHihFJTUxUeHq6aNWuqU6dOuvbaa3XjjTcqIiKixOv9/PPPevPNN7V69WqdOHFCVapUUceOHTV27NhSzcYD83jiiSf05JNPSjo/hqGkyhA/8/5h1apVWrVqldatW6eDBw8qJSVFaWlpioqKUoMGDdSjRw+NHj1al19+eamux3P3LBIGAAAAAG4xSxIAAAAAt0gYAAAAALhFwgAAAADALRIGAAAAAG6RMAAAAABwi4QBAAAAgFskDAAAAADcImEAAAAA4BYJAwAAAAC3SBgAAAAAuEXCAAAAAMAtEgYAAAAAbpEwAAAAAHCLhAEAAACAWyQMAAAAANwiYQAAAADgFgkDAKBCe+KJJ2SxWGSxWHwdCgD4JRIGAKiEDh8+bP8QfTF/AAAVHwkDAMCr5s2bZ084Dh8+7OtwKrSVK1fa/61Xrlzp63AA+KkgXwcAAPC+evXqaceOHW6Pd+jQQZLUtWtXzZ0711thAQBMiIQBACqh4OBgtW/fvsR2kZGRpWoHAKi46JIEAAAAwC0SBgBAmdlsNn300UcaPHiwateurZCQENWoUUP9+vXTzJkzlZub63JOQX/6sWPH2vc1adLEZSC1c1/7X375RY8//rj69u1rv1dMTIzatm2re++9V7t27fL027U7d+6cZsyYof79+zvEEh8fr4kTJ2rt2rVuzz158qQef/xxxcfHq0qVKgoLC1Pjxo11xx13aM2aNSXe+8cff9Qtt9yiJk2aKDw8XBEREWrUqJEuu+wyPfzww/rxxx/tbQsGtffr18++r1+/fi7/1vPmzbuofw8AlYQBAIATSYYko0+fPi7HUlNTjZ49e9rbFPWnTZs2xuHDhx3OW7FiRbHnFPxZsWKF/Zy5c+eW2D4wMNB488033b6X6dOn29tejO+//96Ii4srMZ6ifPfdd0ZMTEyx5913331Gfn5+kec/+OCDJd63evXq9vaHDh0q1b/13LlzL+rfBEDlwBgGAECp5efn69prr9W6deskSX369NGECRPUpEkTJSQk6L333tOiRYu0e/duDRgwQFu3blVUVJQkqVu3btqxY4cWL16sxx9/XJL03XffqW7dug73aNKkiX3barWqatWqGjZsmK644gq1aNFCkZGRSkhI0ObNm/Xaa68pJSVFEyZMUOvWrdW/f3+PvO8VK1bommuukdVqVWBgoO644w4NGzZMDRs2VHZ2tnbt2qVvvvlGS5YscTl369atGjp0qHJzcxUcHKwJEybouuuuU2RkpLZs2aLnnntOhw4d0ptvvqnIyEg9//zzDud/9dVXeuWVVyRJl1xyie699161adNGsbGxOnPmjH777Tf98MMPWr9+vf2cgkHtGzZs0Lhx4yRJ7733nrp16+Zw7fr165fzvxSACsnXGQsAwHzkpsLwxhtv2I+NGjXKsNlsLudOnTrV3mby5MkuxwtXDQ4dOlRsHMeOHTMyMjLcHj9z5oxxySWXGJKMXr16FdnmYisMWVlZRt26dQ1JRkREhEMFxNkff/zhsq9bt272Ssh3333ncvzUqVNG27ZtDUlGQECAsXPnTofjd9xxhyHJaNSokXHu3Dm3905NTXXZV7iqU1zcAFAcxjAAAErtzTfflCTVqFFDb7zxRpGLtz355JNq3bq1JGn27NnKycm54PvVq1dPERERbo/HxsbqqaeekiStWbNGqampF3wvdz744AMlJCRIkp555hn17dvXbdsGDRo4vF6/fr02bNggSbrrrrs0aNAgl3OqVq2qWbNmSTo/NmTmzJkOx5OSkiRJnTt3tldrilKtWrWS3wwAXAASBgBAqSQkJGj37t2SpBtvvFHR0dFFtgsKCrIPbD59+rQ2b95cbjFkZGTo8OHD+u2337Rz507t3LlTwcHB9uPbtm0rt3sV+OqrrySdn2L2rrvuKtO5P/zwg317/Pjxbtv17NlTbdq0cTlHkurUqSNJ+umnn3TgwIEy3R8AygMJAwCgVHbu3GnfvvTSS4ttW/h44fMuREpKiqZOnapWrVopOjpaTZo0Ufv27dWhQwd16NBBQ4YMcWhb3rZs2SJJ6tKlS7HVjqIUvPeQkBB16tSp2LYF/2a///67wyxTo0aNkiSlpqaqffv2uvnmmzV37lzt37+/TLEAwIUiYQAAlMqpU6fs2zVr1iy2be3atYs8r6w2bdqk1q1b69lnn9W+fftkGEax7bOysi74Xu4UJCEF3/SXRcF7r1atmoKCip9npODfzDAMnT592r5/wIABeuONNxQeHq7s7Gx9+umnGjdunFq0aKH69evrnnvu8UhlBQAKkDAAAMqsqLEL5S03N1c33nijUlNTFRwcrIceekirVq1SYmKisrOzZRiGDMNw6KZTUkLhKxf773Xffffp8OHDevnllzV48GDFxsZKko4fP6533nlH8fHx9pmnAKC8kTAAAEql8KDaEydOFNu2YKCu83ll8eOPP+rgwYOSpJkzZ2rGjBm64oorVLt2bYWGhtrbXUwFozTi4uIkSYmJiWU+t+C9p6amymq1Ftu24N/MYrGoatWqLsdr1qypBx98UEuXLtWpU6e0adMmPf7446pSpYoMw9DTTz+txYsXlzlGACgJCQMAoFTat29v3/7111+LbVt4TYDC50ml/7b9t99+s2/fdNNNbttt3LixVNe7UJ07d7bfJzMzs0znFrz33Nxcbd26tdi2Bf9mLVq0UEhISLFtAwIC1LlzZ/3rX//S8uXL7fsXLFjg0M4blSAAFR8JAwCgVOrWrWufyWfBggVKT08vsl1+fr7mzZsn6fyUoQUfuAuEhYXZt4ubcrXwN/IZGRlFtrHZbJo9e3ap4r9QQ4cOlSRlZmbapz8trYEDB9q333vvPbft1q1bp127drmcUxqdO3e2VyScB32X9t8aAIpDwgAAKLX77rtPknTy5Endf//9RbZ58skn7R9+77rrLofuQ5Lj4OHipglt0aKFfbsgAXE2ZcqUcp22tSi333676tWrJ0n6xz/+oVWrVrlte+zYMYfX3bt3V9euXSWdX5OicDWgwNmzZ3X33XdLOl85uPfeex2Of/rpp8UO5t64caN9kHThVbKl0v9bA0BxLIZZR4gBAHymoCtLnz59tHLlSvv+/Px89e7dW+vWrZMk9e/fX3/729/UpEkTJSYm6r333tPnn38uSWrWrJm2bt3qstjYuXPnVLNmTWVnZ6tz58567rnn1KhRIwUEnP8Oq169egoPD1dGRoaaNm2q5ORkBQYG6s4779SIESMUFxen/fv32z+A9+zZU2vXrpUkzZ07V2PGjHG43xNPPKEnn3xS0oUPil6xYoUGDRokq9WqoKAg3XHHHRo+fLjq16+vnJwc7dmzR19//bW+/PJLl2/yt27dqksvvVS5ubkKCQnRxIkTNXToUEVGRmrLli167rnn7GM1Jk+erOeff97h/MaNG+vs2bMaNmyYrrjiCrVs2VKRkZFKTU3VmjVr9Prrr+vUqVMKDAzUL7/8Yk9QCjRo0EDHjh1TkyZN9Morr6hVq1YKDAyUJNWqVcvtehoAYOezNaYBAKYlyZBk9OnTx+VYamqq0bNnT3ubov60adPGOHz4sNvrT5482e25K1assLf79ttvjbCwMLdt+/bta+zcudP+eu7cuS73mj59uv34xfj222+NqlWrFvu+3d3ju+++M2JiYoo977777jPy8/Ndzm3UqFGJ9wwNDS3yvRuGYcycOdPtee7OAYDC6JIEACiTatWq6aefftIHH3ygq6++WrVq1VJwcLCqV6+uvn376o033tDWrVvVqFEjt9d47rnnNHv2bPXu3VvVqlWzf+Pt7KqrrtLGjRt1++23q27dugoODlaNGjXUp08fzZo1S8uXL1dkZKSn3qpLLAcPHtQzzzyjHj16qHr16goMDFRMTIw6d+6sBx980GGwd2GDBg3S/v37NXXqVHXq1EkxMTEKDQ1Vw4YNddttt2n16tV644037FWWwlasWKFXX31VN9xwgzp06KAaNWooKChIMTExio+P18MPP6xdu3a5VFYK3Hvvvfrvf/+rQYMGqWbNmiWuBwEAzuiSBAAAAMAtKgwAAAAA3CJhAAAAAOAWCQMAAAAAt0gYAAAAALhFwgAAAADALRIGAAAAAG6RMAAAAABwi4QBAAAAgFskDAAAAADcImEAAAAA4BYJAwAAAAC3SBgAAAAAuEXCAAAAAMAtEgYAAAAAbpEwAAAAAHCLhAEAAACAWyQMAAAAANwiYQAAAADgFgkDAAAAALdIGAAAAAC4RcIAAAAAwC0SBgAAAABukTAAAAAAcIuEAQAAAIBbJAwAAAAA3Pp/q+TUfDaFnUIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1, 1, figsize=(4, 3), dpi=200)\n",
    "ax.plot(\n",
    "    cost_AGP.cpu()[9:],\n",
    "    best_seen_AGP.cpu()[9:],\n",
    "    label=\"AGP\"\n",
    ")\n",
    "ax.plot(\n",
    "    cost_MES.cpu()[9:],\n",
    "    best_seen_MES.cpu()[9:],\n",
    "    label=\"MES\"\n",
    ")\n",
    "\n",
    "ax.set_title(\"Branin\", fontsize=\"12\")\n",
    "ax.set_xlabel(\"Total cost\", fontsize=\"10\")\n",
    "ax.set_ylabel(\"Best seen\", fontsize=\"10\")\n",
    "ax.tick_params(labelsize=10)\n",
    "ax.legend(loc=\"lower right\", fontsize=\"7\", frameon=True, ncol=1)\n",
    "plt.tight_layout()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
